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Preface 


Continuing  concern  about  large  cost  overruns  in  a  broad  range  of  major  defense  pro¬ 
grams  led  Congress  to  enact  new  statutory  provisions  extending  the  ambit  of  the  exist¬ 
ing  Nunn-McCurdy  Act,  stipulating  that  the  U.S.  Department  of  Defense  (DoD) 
review  and  report  on  the  factors  affecting  program  costs  in  both  specific  and  general 
terms.  In  accordance  with  the  revised  Nunn-McCurdy  Act,  the  Performance  Assess¬ 
ments  and  Root  Cause  Analysis  (PARCA)  office  must  provide  its  root  cause  explana¬ 
tion  as  part  of  a  60-day  program  review  triggered  when  the  applicable  military  depart¬ 
ment  secretary  reports  a  breach. 

In  March  2010,  in  view  of  staffing  limitations,  the  newly  created  PARCA  within 
the  Office  of  the  Secretary  of  Defense  (OSD)  elected  to  rely  on  federally  funded 
research  and  development  center  support  in  discharging  its  new  responsibilities.  Since 
then,  PARCA  engaged  the  RAND  Corporation  to  conduct  multiple  studies  on  the 
root  causes  of  Nunn-McCurdy  breaches  or  other  large  cost  increases  in  seven  major 
defense  acquisition  programs:  the  Wideband  Global  Satellite,  the  Longbow  Apache, 
the  Zumwalt-c\a.ss  destroyer  (DDG-1000),  the  Joint  Strike  Fighter,  the  Excalibur,  the 
Joint  Tactical  Radio  System  Ground  Mobile  Radio,  and  the  Navy  Enterprise  Resource 
Planning.1 

This  report  derives  management  perspectives  from  analysis  of  Nunn-McCurdy 
breaches  and  other  cost  growth  questions  with  three  issues  related  to  the  acquisition  of 
materiel  by  DoD.  Partly  in  response  to  a  finding  by  the  U.S.  Government  Accountabil¬ 
ity  Office  that  the  tenure  of  program  managers  was  relatively  short,  the  head  of  PARCA 
asked  RAND  to  analyze  the  tenure  of  program  managers  and  to  provide  an  alternative 
perspective  on  the  data  relating  to  this  topic,  along  with  any  conclusions  that  could  be 


1  See  Irv  Blickstein,  Michael  Boito,  Jeffrey  A.  Drezner,  James  Dryden,  Kenneth  Horn,  James  G.  Kallimani, 
Martin  C.  Libicki,  Megan  McKernan,  Roger  C.  Molander,  Charles  Nemfakos,  Chad  J.  R.  Ohlandt,  Caroline 
Reilly,  Rena  Rudavsky,  Jerry  M.  Sollinger,  Katharine  Watkins  Webb,  and  Carolyn  Wong,  Root  Cause  Analyses 
of  Nunn-McCurdy  Breaches,  Volume  1:  Zumwalt-Class  Destroyer,  Joint  Strike  Fighter,  Longbow  Apache,  and  Wide¬ 
band  Global  Satellite,  Santa  Monica,  Calif.:  RAND  Corporation,  MG-1171/1-OSD,  2011;  and  Irv  Blickstein, 
Jeffrey  A.  Drezner,  Martin  C.  Libicki,  Brian  Mclnnis,  Megan  McKernan,  Charles  Nemfakos,  Jerry  M.  Sollinger, 
and  Carolyn  Wong,  Root  Cause  Analyses  of  Nunn-McCurdy  Breaches,  Volume  2:  Excalibur  Artillery  Projectile  and 
the  Navy  Resource  Planning  Program,  with  an  Approach  to  Analyzing  Complexity  and  Risk,  Santa  Monica,  Calif.: 
RAND  Corporation,  MG-1171/2-OSD,  2012. 
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drawn  about  whether  recent  policy  changes  have  been  effective  in  increasing  tenure.  A 
second  issue  was  the  management  of  Acquisition  Category  II  programs  and  whether 
existing  decentralized  systems  used  to  track  the  cost  growth  and  performance  of  these 
programs  are  adequate  or  whether  additional  centralized  guidance  is  warranted.  The 
third  issue  that  this  report  deals  with  is  an  exploratory  one  to  determine  whether  it  is 
feasible  in  acquisition  programs  to  identify  program  assumptions  that  are  so  key  to  the 
program’s  success  that  they  could  be  used  as  a  way  to  manage  cost  and  schedule  risk. 

This  report  should  interest  DoD  staff  and  military  personnel  who  are  involved  in 
the  acquisition  of  defense  systems. 

This  research  was  sponsored  by  OSD  PARCA  and  conducted  within  the  Acquisi¬ 
tion  and  Technology  Policy  Center  of  the  RAND  National  Defense  Research  Insti¬ 
tute,  a  federally  funded  research  and  development  center  sponsored  by  the  Office  of  the 
Secretary  of  Defense,  the  Joint  Staff,  the  Unified  Combatant  Commands,  the  Navy, 
the  Marine  Corps,  the  defense  agencies,  and  the  defense  Intelligence  Community. 

For  more  information  on  the  RAND  Acquisition  and  Technology  Policy  Center, 
see  http://www.rand.org/nsrd/ndri/centers/atp.html  or  contact  the  director  (contact 
information  is  provided  on  the  web  page). 
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Summary 


In  light  of  continuing  program  cost  growth  and  observations  by  the  U.S.  Government 
Accountability  Office  (GAO)  placing  defense  acquisition  on  the  high-risk  target  list, 
Congress  has  become  particularly  concerned  about  the  execution  of  major  defense 
acquisition  programs.  This  concern,  coupled  with  the  reality  of  shrinking  defense  bud¬ 
gets,  led  Congress  to  enact  statutory  provisions  that  would  focus  greater  policymaker 
attention  on  the  oversight  of  major  defense  acquisition  programs  (MDAPs)  and  other 
large,  costly  programs.2  For  example,  the  Weapon  Systems  Acquisition  Reform  Act  of 
2009  (WSARA)  established  a  number  of  requirements  affecting  the  operation  of  the 
Defense  Acquisition  System  and  the  duties  of  the  key  officials  who  support  it,  includ¬ 
ing  the  requirement  to  establish  a  new  organization  in  the  Office  of  the  Secretary  of 
Defense  (OSD)  with  the  mandate  to  conduct  and  oversee  performance  assessments 
and  root  cause  analyses  for  MDAPs.3 

In  March  2010,  the  director  of  the  Office  of  Performance  Assessments  and  Root 
Cause  Analysis  (PARCA)  determined  that  he  required  support  to  execute  his  statutory 
responsibilities  and  turned  to  federally  funded  research  and  development  centers  and 
academia  to  help  with  the  research  and  analysis  of  program  execution  status.  RAND 
was  among  the  institutions  engaged  to  carry  out  root  cause  analyses,  which  it  has  com¬ 
pleted  for  six  programs  to  date.4 

In  addition  to  the  root  cause  analysis  of  Nunn-McCurdy  breaches  in  specific 
acquisition  programs,  the  PARCA  director  posed  some  additional  questions  to  RAND 
to  determine  whether  they  affect  the  management  of  such  programs  or  might  provide  a 
useful  perspective  in  managing  them.  One  pertained  to  program  manager  (PM)  tenure, 
which  was  not  a  featured  cause  in  the  analyses  RAND  had  previously  performed. 
However,  PARCA  asked  RAND  to  calculate  current  PM  tenure  using  easily  available 
sources,  in  part  to  determine  whether  tenure  periods  have  increased  since  policy  guid¬ 
ance  designed  to  lengthen  tenure  was  published  in  2005  and  2007.  A  second  question 
posed  was  whether  existing  decentralized  systems  used  to  track  the  cost  growth  and 


2  Public  Law  111-383,  Ike  Skelton  Defense  Authorization  Act  for  Fiscal  Year  2011,  December  20,  2010. 

3  Public  Law  111-23,  Weapon  Systems  Acquisition  Reform  Act  of  2009,  May  22,  2009. 

Blickstein  et  al.,  2011,  2012. 
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performance  of  acquisition  category  (ACAT)  II  programs  are  sufficient,  or  whether 
additional  centralized  guidance  and  control  from  OSD  are  warranted.  Additional 
oversight  may  provide  transparency  of  ACAT  II  performance  and  contribute  to  more 
efficient  acquisition  processes.  However,  new  reporting  and  control  requirements  will 
place  additional  burden  on  the  defense  agencies  and  military  departments  as  well  as  on 
the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logis¬ 
tics  (OUSD  (AT&L)  and  the  PARCA  office,  which  is  given  the  authority  to  issue  guid¬ 
ance  on  ACAT  II  programs  by  WSARA  and  will  bear  responsibility  for  any  additional 
oversight.  The  third  question  dealt  with  the  management  of  cost  and  schedule  risk  and 
whether  the  identification  of  key  assumptions,  which  we  call  framing  assumptions, 
could  be  a  useful  risk  management  tool. 


Program  Manager  Tenure 

Program  manager  tenure  is  frequently  mentioned  in  regard  to  improving  acquisition 
outcomes  and  accountability.  Policies  have  attempted  to  enforce  longer  program  man¬ 
ager  tenure  over  the  past  few  decades,  because  it  has  been  found  in  at  least  one  study 
that  longer  program  tenure  is  one  of  the  building  blocks  for  program  success  typically 
leading  to  lower  cost  growth.5  GAO  reported  in  2007  that  PM  tenure  was  17.2  months 
in  the  programs  it  reviewed. 

PARCA  asked  RAND  to  calculate  current  PM  tenure  using  easily  available 
sources  to  provide  an  understanding  of  the  length  of  PM  tenure.  Part  of  the  motiva¬ 
tion  behind  this  request  was  to  see  if  tenure  periods  have  been  increasing  since  policy 
guidance  designed  to  lengthen  tenure  was  published  in  2005  and  2007. 

Results 

To  quantify  PM  tenure  using  current  data,  we  extracted  program  point-of-contact  data 
from  Selected  Acquisition  Reports  (SARs)  from  calendar  years  1997  through  2011. 
Our  final  database,  excluding  incomplete  tenure  periods,  contained  370  program  man¬ 
ager  tenure  periods  from  136  programs  submitting  SARs  from  1997  to  2011  (both 
annual  and  quarterly). 

Using  the  largest  available  dataset  of  program  manager  tenure  periods  that  had 
been  completed,  we  calculated  that  the  average  tenure  for  those  program  manager  peri¬ 
ods  was  33.7  months,  which  aligns  more  closely  to  the  GAO  statement  in  2005  regard¬ 
ing  PM  tenure  (no  more  than  three  to  four  years)  rather  than  the  November  2007 
statement  of  17.2  months.  The  results  of  this  set  of  data  are  presented  in  Figure  S.l.  We 
also  calculated  tenure  by  eliminating  several  outliers  in  terms  of  tenure  length,  and  we 


5  1st  Lt  Christina  F.  Rusnock,  Predicting  Cost  and  Schedule  Growth  for  Military  and  Civil  Space  Systems,  thesis, 
Wright-Patterson  Air  Force  Base,  Ohio:  Air  Force  Institute  of  Technology,  AFIT/GRD/ENC/08M-01,  March 
2008,  p.  99. 
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Figure  S.1 

Average  Number  of  Months  PM  Is  Assigned  to  Program  (full  dataset) 


1990  1992  1994  1996  1998  2000  2002  2004  2006  2008  2010  2012 


Year  PM  assigned  to  program 

aRAND  calculated  average  of  370  program  manager  terms  using  1997  to  201 1  SAR  data. 
SOURCE:  SARs  (1997-2011). 
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included  only  tenure  periods  that  started  between  1997  and  2008.  The  result  was  an 
average  of  33.2  months,  only  slightly  lower  than  the  average  from  the  larger  dataset. 

This  initial  data  analysis  did  not  help  us  answer  the  question  regarding  PM  tenure 
periods  after  the  2005  and  2007  PM  tenure  policies,  however,  because  there  were 
too  many  tenure  periods  that  had  not  been  completed  after  2008.  We  therefore  also 
conducted  a  statistical  analysis  that  includes  open  tenure  periods.  One  challenge  in 
introducing  open  tenure  periods  to  this  analysis  is  that  these  periods  cause  bias  in  the 
average,  because  the  program  managers  have  not  finished  their  tenure  in  a  particular 
program,  which  makes  the  overall  average  shorter  than  if  they  had.  We  applied  a  sta¬ 
tistical  approach  that  calculates  probable  tenures  for  open  periods.  (See  the  discussion 
in  Chapter  Two  for  details). 

Given  this  statistical  treatment,  the  data  provide  some  support  that  program 
managers  remained  in  positions  longer  after  the  release  of  the  revised  guidelines  in 
2007.  However,  because  of  the  small  numbers  of  positions  and  the  large  number  of 
open  positions  among  those  beginning  in  2006  or  later,  the  differences  among  groups 
are  not  statistically  significant,  and  the  observed  differences  could  be  chance  occur¬ 
rences  rather  than  real  difference  among  the  groups  of  positions. 

Additional  years  of  data  for  these  positions  and  new  positions  starting  in  2011 
or  later  could  help  to  establish  if  the  differences  among  groups  demonstrate  persistent 
changes  to  practice  after  the  release  of  the  revised  guidelines.  But  even  if  the  differences 
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proved  to  be  persistent,  they  might  not  have  been  caused  by  the  revised  guidelines. 
We  would  need  to  rule  out  other  potential  sources  of  change  before  we  could  make 
that  attribution.  However,  analysis  of  additional  data  would  be  a  valuable  first  step  to 
exploring  fully  the  effects  of  the  2007  guideline  change. 

Observations 

Our  research  leads  us  to  the  following  observations: 

The  intent  of  policies  to  lengthen  PM  tenure  may  not  have  been  achieved. 

PM  tenure  has  been  reviewed/quantified  periodically  during  the  last  40  years.  It  is  dif¬ 
ficult  to  assess  whether  these  policies  have  been  successful  based  on  previous  data  and 
studies  and  current  data. 

No  enforcement  mechanism  has  been  readily  apparent  over  time.  This  could 
be  because  enforcement  is  limited  because  of  the  fundamental  conflict  that  exists 
between  what  military  officers  need  to  do  to  be  promoted  and  their  tenure  as  program 
managers.  Unless  these  two  are  aligned  such  that  lengthy  tenure  in  a  program  can  be 
advantageous  for  promotion,  then  it  appears  unlikely  that  these  tenure  policies  will 
consistently  yield  positive  results. 

We  cannot  determine  whether  the  policies  of  2005  and  2007  have  helped  to 
lengthen  PM  tenure.  A  statistical  analysis  using  open/closed  tenure  periods  indicated 
that  the  estimate  of  the  mean  time  in  position  for  tenure  periods  starting  after  2007 
is  biased,  because  of  the  large  number  of  tenure  periods  that  remain  open  at  the  last 
data  collection.  This  creates  an  average  that  is  lower  than  if  all  PM  tenure  periods  in 
a  sample  have  both  a  beginning  and  an  ending.  Using  these  same  data,  we  found  that 
PMs  are  less  likely  to  stay  as  long  in  their  tenure  periods,  meaning  that  75-85  percent 
will  likely  reach  two  years,  but  only  50  percent  will  likely  reach  three  years.  PMs  that 
started  in  tenure  periods  after  the  policy  change  are  much  more  likely  to  remain  in 
those  periods  longer  than  PMs  starting  in  periods  between  2005  and  2007. 

We  also  found  that  adjusting  for  a  variety  of  data  issues  (outliers,  open  periods, 
etc.)  will  give  us  more  confidence  in  the  data  but  will  only  minimally  change  the  result 
of  PM  tenure  during  the  last  14  years  for  MDAPs. 

By  taking  into  account  closed  program  manager  tenure  periods,  we  found 
that  PM  tenure  is  on  average  33  months.  This  result  is  much  higher  than  GAO’s 
2007  figure  of  17.2  months  for  39  programs  but  includes  a  larger  sample  size  covering 
more  years  and  does  not  include  any  open  periods. 

In  conclusion,  this  analysis  has  been  able  to  quantify  PM  tenure  using  current 
data  but  cannot  definitely  say  whether  recent  policies  regarding  PM  tenure  have  had 
any  positive  effect  toward  lengthening  tenure  over  the  last  several  years,  because  there 
are  still  too  many  open  tenure  periods  during  that  time  period. 
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Performance  of  ACAT  II  Programs 

Because  additional  reporting  and  control  requirements  come  with  costs,  the  decision  to 
mandate  additional  guidelines  for  ACAT  II  programs  may  be  warranted  only  if  there 
are  problems  with  the  existing  system  that  result  in  cost  growth  and  schedule  slippage 
and  that  can  be  dealt  with  by  more  centralized  oversight.  To  determine  if  such  perfor¬ 
mance  issues  do  plague  current  ACAT  II  programs,  we  conducted  two  sets  of  analyses, 
one  on  a  sample  of  ACAT  II  programs  and  one  on  a  comparable  set  of  MDAPs.  Both 
sets  include  programs  from  across  military  departments  and  procurement  program 
categories  (e.g.,  aircraft,  weapons,  shipbuilding  and  conversion).  We  evaluate  program 
performance  focusing  specifically  on  unit  cost  growth  over  the  program’s  life  and  insta¬ 
bility  in  annual  quantity  procured  over  time.  Then,  we  compared  the  overall  perfor¬ 
mance  of  the  ACAT  II  and  MDAP  samples.  This  comparison  leads  to  some  general 
inferences  about  the  performance  of  ACAT  II  programs  under  the  current  system  of 
oversight,  their  performance  relative  to  MDAPs,  which  are  subject  to  more  centralized 
monitoring  and  requirements,  and  the  need  for  additional  centralized  reporting  and 
control  requirements. 

For  our  analysis,  we  rely  on  budget  information  provided  by  the  military  depart¬ 
ments  and  included  in  the  President’s  official  budget  justifications  as  well  as  congressio¬ 
nal  hearings  and  testimony  to  construct  program  narratives.  The  information  includes 
summary  information  on  cost  and  quantity  for  the  past,  current,  and  requested  fiscal 
year.  We  used  the  information  to  select  ten  ACAT  II  programs  and  seven  MDAP  pro¬ 
grams,  covering  all  procurement  program  categories  across  all  three  military  depart¬ 
ments.  We  used  the  information  to  construct  time  series  datasets  of  unit  cost  and 
annual  quantity  procured  for  each  program  in  our  sample  by  collecting  and  integrating 
annual  data  on  cost  and  quantity  provided  in  these  budget  documents. 

Using  the  time  series  data  on  program  cost  and  quantity,  we  graphed  annualized 
unit  cost  against  annualized  quantity.  We  assessed  these  graphs  looking  at  large  revi¬ 
sions  and  sharp  fluctuations  in  quantity  and  cost  as  well  as  slow  growth  over  time.  To 
explain  cost  growth  or  periods  of  apparently  weak  performance,  we  matched  the  nar¬ 
rative  information  on  each  program  to  the  unit  cost  graphs. 

Results 

Although  our  sample  size  and  composition  prevent  a  meaningful  statistical  compari¬ 
son  of  cost  growth,  our  qualitative  assessment  of  program  performance  should  have 
some  general  application  to  acquisition  programs.  Furthermore,  our  intention  was 
to  conduct  broad  assessment  of  the  performance  of  the  two  program  types  to  deter¬ 
mine  whether  ACAT  II  programs  appear  to  perform,  on  average,  better  or  worse  than 
MDAPs,  rather  than  to  provide  precise  metrics  on  this  performance  or  precise  quanti¬ 
fication  unit  cost  growth  over  time.  We  find  that,  overall,  both  ACAT  II  and  MDAP 
programs  perform  reasonably  well  once  they  have  entered  production.  In  both  cases, 
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programs  experience  some  cost  growth  and  some  instability  in  unit  cost  and  annual 
quantity  procured  over  the  period  considered.  However,  in  neither  case  do  we  observe 
crippling  instability,  runaway  cost  growth,  or  severe  production  delays.  Of  the  ACAT 
II  programs  we  consider,  we  find  that  four  show  no  reason  for  concern,  five  have  some 
issues  of  minor  concern,  and  three  warrant  some  more  significant  concerns.  Across  the 
board,  ACAT  II  programs  in  our  sample  are  more  significantly  affected  by  instability 
in  unit  cost  than  by  actual  cost  growth.  Instability  is  particularly  likely  early  in  the 
lives  of  new  ACAT  II  programs,  associated  with  development  and  modernizations. 
There  also  seems  to  be  a  clear  relationship  between  unit  cost  and  quantity.  Much  of 
the  cost  growth  that  we  do  observe  occurs  as  a  result  of  downward  revisions  in  pro¬ 
curement  quantities,  perhaps  stemming  from  changes  in  demand  associated  with  the 
contingencies  in  Iraq  and  Afghanistan. 

Programs  in  the  MDAP  sample  can  similarly  be  distributed  across  the  three  per¬ 
formance  categories:  no  serious  concerns,  minor  concerns,  and  some  concerns.  Our 
assessment  places  three  programs  in  the  first  category,  three  in  the  second,  and  only 
one  in  the  third.  As  was  true  for  ACAT  II  programs,  several  MDAP  programs  show 
some  cost  growth  over  their  program  life  (including  projected  out-years),  and  some 
have  experienced  periods  of  instability  associated  with  modernizations  and  fluctua¬ 
tions  in  demand.  However,  several  also  show  a  decrease  in  average  unit  cost  over  the 
period  considered. 

Our  assessment  suggests  that  ACAT  IIs  reach  a  level  of  performance  currently, 
without  rigorous  centralized  oversight,  that  is  at  least  equal  to  that  of  MDAPs  operat¬ 
ing  with  centralized  reporting  and  control  requirements  and  oversight. 


Framing  Assumptions 

Defense  acquisition  programs  routinely  must  estimate  cost,  schedule,  and  technologi¬ 
cal  performance  far  in  advance  of  actual  work.  And  they  must  account  for  differ¬ 
ences  in  acquisition  strategy  and  market  conditions.  As  a  result,  programs  must  make 
assumptions  about  their  programs  and  the  conditions  that  might  affect  them.  When 
these  assumptions  prove  faulty,  they  can  cause  the  program  to  miss  important  cost  and 
schedule  benchmarks,  which  can  lead  to  breaches  of  Nunn-McCurdy  thresholds.  Key 
assumptions  are  called  “framing”  because  of  the  influence  they  have  on  program  per¬ 
formance.  This  exploratory  research  was  done  in  an  attempt  to  define  framing  assump¬ 
tions  in  a  way  that  others  can  use  them  to  assess  and,  potentially,  control  program  risk 
and  to  explore  the  possibility  of  identifying  them  for  a  selected  set  of  programs. 

Results 

We  arrived  at  the  following  definition  for  a  framing  assumption: 
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A  framing  assumption  is  any  explicit  or  implicit  assumption  that  is  central  in  shaping 
cost,  schedule,  and/or  performance  expectations. 

A  framing  assumption  has  the  following  five  characteristics.  First,  the  conse¬ 
quences  of  the  assumption  being  wrong  will  significantly  affect  the  program  in  a  way 
that  matters  (e.g.,  significant  cost  growth  or  schedule  slippage).  In  other  words,  the 
assumption  is  important  to  the  success  of  the  program.  Second,  the  consequences  of 
the  assumption  being  wrong  cannot  be  avoided.  Third,  the  outcome  or  certainty  with 
respect  to  the  assumption  is  unknown  (there  is  some  risk).  Fourth,  the  consequences 
of  the  assumption  failing  or  holding  true  do  not  hinge  on  other  events  or  chain  of 
events.  Finally,  a  framing  assumption  should  typically  distinguish  a  program  from 
all  other  programs.  An  example  of  a  framing  assumption  might  be  that  “competitive 
prototyping  will  save  5  percent  of  the  procurement  cost.”  This  would  be  opposed  to  an 
assumption  that  held,  “the  contractor  will  perform  well,”  which  would  be  common  to 
all  programs. 

Researchers  then  examined  five  defense  programs  in  various  stages  of  maturity: 

•  Advanced  Pilot  Training  (APT)  System 

•  Joint  Light  Tactical  Vehicle 

•  Joint  Precision  Approach  and  Landing  System 

•  Littoral  Combat  Ship  Modules 

•  Space  Fence. 

For  each  program,  we  attempted  to  identify  framing  assumptions  in  three  cat¬ 
egories:  technological,  management  incentives  and  program  structures,  and  mission 
requirements.  The  results  appear  in  Table  S.l. 

A  similar  exercise  using  the  seven  programs  on  which  RAND  conducted  a  root 
cause  analysis  yielded  similar  results,  i.e.,  researchers  were  able  to  apply  the  definition 
to  the  programs  and  identify  framing  assumptions.  Note,  however,  that  identifying 
framing  assumptions  is  typically  much  easier  after  the  fact,  and  the  reason  for  the 
root  cause  analyses  was  that  the  programs  had  already  breached  the  Nunn-McCurdy 
thresholds. 

The  important  finding  of  this  research  is  that  it  is  possible  to  define  framing 
assumptions  and  apply  them  to  programs  that  are  in  their  early  stages,  which  is  when 
program  managers  would  want  to  do  this  to  help  them  manage  program  risk.  The 
efficacy  of  using  framing  assumptions  to  manage  risk  remains  to  be  seen.  However, 
this  research  suggests  some  common  characteristics  for  such  assumptions.  One  is  that 
the  assumption  needs  to  focus  at  a  high  level  and  not  on  the  fine-grained  detail  of  the 
program.  Additionally,  framing  assumptions  should  be  relatively  few  in  number;  about 
three  to  five  seems  right.  Clearly,  more  assumptions  could  be  identified,  but  the  focus 
must  fall  on  the  ones  that  can  truly  affect  the  program’s  outcomes.  Assumptions  need 


Table  S.1 

Framing  Assumptions  Identified  for  Five  Acquisition  Programs 


Program 

Technological 

Management  Incentives 
and  Program  Structures 

Mission  Requirements 

APT  System 

Training  aircraft  (both  airframe  and  avionics) 
will  be  nondevelopmental. 

Potential  customers  will  want  to  buy  the  APT 
with  minimal  modifications. 

Current  training  scenario  is  valid: 

Use  of  existing  T-38  can  be 
extended  until  2020. 

Simulators  can  be  used  instead  of 
actual  flight  time  (to  save  money). 

Joint  Light 
Tactical 

Vehicle 

Incremental/open  architecture  will  reduce  risk  and 
allow  for  more  efficient  upgrades. 

Competitive  prototyping  will  reduce  risk  and  cost. 

A  Joint  Army  and  Marine  Corps  program  will  save 
money  as  requirements  are  compatible. 

The  services  have  effectively  assessed  long¬ 
term  and  short-term  needs. 

Joint  Precision 
Approach  and 
Landing  System 

Incremental  development  will  lower  risk,  while  COTS/ 
GOTS  hardware  and  software  will  lower  costs. 

Navy  is  the  best  service  to  lead  acquisition  with 
the  ordering  of  the  increments  driven  by  the 
Navy's  more  demanding  requirements. 

System  is  suitable  for  all  types  of  air 
vehicles. 

Littoral  Combat 
Ship  Modules 

Sea  frames  and  modules  can  be  independently 
developed: 

Spiral/incremental  development  will  lower  risk. 

Can  successfully  test  modules  on  other  ship 
platforms. 

New  capabilities  can  be  added  easily. 

New,  open  business  model  approach  allows  for 
independent  development  of  sea 
frames  and  modules: 

Leveraging  PARMs  and  other  programs. 

Benefits  of  open  architecture/commercial 
practices. 

Government  suited  to  act  as  system 
integrator. 

The  Navy  is  willing  to  drop  requirements 
(in  spiral  context)  to  keep  to  schedule. 

RDA  can  be  program  focal  point  (4  PMs, 

3  PEOs,  2  SYSCOMs). 

Space  Fence 

Capability  is  achievable  despite  some  immature 
technologies  at  the  outset. 

Block  approach  is  a  more  effective  acquisition 
strategy. 

Minimal  manpower  required  to  operate 
and  support  system. 

Can  scale  technology  to  track  order  of  magnitude 
greater  number  of  objects  (radar  components, 
software  interoperability). 

The  legal,  diplomatic,  and  political  issues  with 
site  decisions  can  be  resolved  easily. 

Two  instead  of  three  sites  will  be  sufficient 
to  meet  operational  goals. 

Competitive  prototyping  reduces  risk  and  cost. 
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to  be  revisited  as  the  program  moves  through  the  acquisition  process,  because  they  can 
change  or  even  be  supplanted  by  other  assumptions.  Implicit  assumptions  are  more  dif¬ 
ficult  to  identify  than  the  explicit  ones. 

As  mentioned,  the  utility  of  framing  assumptions  as  a  risk  management  tech¬ 
nique  remains  to  be  demonstrated.  A  useful  extension  of  this  research  would  be  to 
examine  the  assumptions  of  a  broader  range  of  programs  to  identify  those  that  might 
be  problematic. 
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Abbreviations 


AAE 

Army  Acquisition  Executive 

ABV 

assault  breacher  vehicle 

ABP 

assumption-based  planning 

ACAT 

acquisition  category 

ACP 

Army  Cost  Position 

ACTD 

Advanced  Concept  Technology  Demonstration 

ADM 

Acquisition  Decision  Memorandum 

AFRB 

Air  Force  Review  Board 

AFSSS  VHF 

Air  Force  Space  Surveillance  System  Very  Fiigh  Frequency 

AME 

aviation  mission  equipment 

AoA 

Analysis  of  Alternatives 

APB 

Acquisition  Program  Baseline 

APT 

Advanced  Pilot  Training  System 

APUC 

average  procurement  unit  cost 

ASA  (AFT) 

Assistant  Secretary  of  the  Army  (Acquisition,  Eogistics,  and 
Technology) 

ASA  (FMC) 

Assistant  Secretary  of  the  Army  (Financial  Management  and 
Comptroller 

ASA  (I&E) 

ASA  (M&RA) 

ASARC 

Assistant  Secretary  of  the  Army  (Installations  and  Environment) 
Assistant  Secretary  of  the  Army  (Manpower  and  Reserve  Affairs) 
Army  Systems  Acquisition  Review  Council 
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ASN  (RDA) 

Assistant  Secretary  of  the  Navy  (Research,  Development  & 
Acquisition) 

ASP 

Acquisition  Strategy  Panel 

ASR 

Acquisition  Strategy  Review 

ATC 

Air  Traffic  Control 

BCT 

Brigade  Combat  Team 

BY 

base  year 

CAE 

Component  Acquisition  Executive 

CAPE 

Cost  Assessment  and  Program  Evaluation 

CCD 

charge-coupled  device 

CDD 

Capability  Development  Document 

CIO 

chief  information  officer 

CMC 

Commandant  of  the  Marine  Corps 

CNO 

Chief  Naval  Officer 

COCOM 

combatant  commander 

COTS 

commercial  off-the-shelf 

CY 

calendar  year 

DAB 

Defense  Acquisition  Board 

DAES 

Defense  Acquisition  Executive  Summary 

DAMIR 

Defense  Acquisition  Management  Information  Retrieval 

DASA  (ASM) 

DCS 

Deputy  for  Acquisition  and  Systems  Management 

Deputy  Chief  of  Staff 

DDL 

digital  data  link 

DoD 

U.S.  Department  of  Defense 

DoDD 

Department  of  Defense  Directive 

DoDI 

Department  of  Defense  Instruction 

DDR&E 

Director  of  Defense  Research  and  Engineering 

DSB 

Defense  Science  Board 

DT&E 

developmental  test  and  evaluation 
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DUSA  (T&E) 

Deputy  Under  Secretary  of  the  Army  (Test  and  Evaluation) 

ELSG 

Electronic  Systems  Group 

ELSS 

Electronic  Systems  Squadron 

ELSW 

Electronic  Systems  Wing 

EMD 

engineering,  manufacturing,  and  development 

ER 

extended  range 

ERP 

Enterprise  Resource  Planning 

FCS 

Future  Combat  System 

FRP 

full-rate  production 

FY 

fiscal  year 

FYDP 

Future  Years  Defense  Program 

GAO 

U.S.  Government  Accountability  Office 

GOTS 

government  off-the-shelf 

GWOT 

Global  War  on  Terror 

HAC 

House  Appropriations  Committee 

HASC 

House  Armed  Services  Committee 

HBCT 

Heavy  Brigade  Combat  Team 

HIMARS 

High  Mobility  Artillery  Rocket  System 

ICD 

Initial  Capabilities  Document 

ICE 

independent  cost  estimate 

HR 

imaging  infrared 

ILCM 

Integrated  Life  Cycle  Management 

IPT 

Integrated  Product  Team 

IR 

infrared 

ISP 

Information  Support  Plan;  also  Initial  Spares  Package 

JDAM 

joint  direct  attack  munition 

JLTV 

joint  light  tactical  vehicle 

JPALS 

joint  precision  approach  and  landing  system 

JROC 

Joint  Requirements  Oversight  Council 
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JSF 

JSOF 

JTRS 

KPP 

Joint  Strike  Fighter 

Joint  Special  Operations  Force 

Joint  Tactical  Radio  System 
key  performance  parameter 

KSA 

key  system  attribute 

LAIRCM 

large  aircraft  infrared  countermeasure 

LCAC  SLEP 

Landing  Craft  Air  Cushion  Service  Life  Extension  Program 

LCMP 

Life  Cycle  Management  Plan 

LCRRPR 

low-cost  reduced-range  practice  rocket 

LCS 

littoral  combat  ship 

LRIP 

low-rate  initial  production 

LSI 

lead  system  integrator 

LUH 

light  utility  helicopter 

MCATTD 

Marine  Corps  Advanced  Technology  Transition  Demonstrator 

MDA 

Milestone  Decision  Authority 

MDAP 

major  defense  acquisition  program 

MDR 

milestone  decision  review 

MLRS  RRPR 

Multiple  Launch  Rocket  System  Reduced-Range  Practice 
Rocket 

MOA 

memorandum  of  agreement 

MP 

mission  package 

MS 

milestone 

MTVR 

medium  tactical  vehicle  replacement 

NATO 

North  Atlantic  Treaty  Organization 

NDAA 

National  Defense  Authorization  Act 

NDI 

nondevelopmental  items 

oco 

Overseas  Contingency  Operation 

OEF 

Operation  Enduring  Freedom 

OIF 

Operation  Iraqi  Freedom 
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OIPT 

overarching  integrated  process  team 

OPEVAL 

operational  evaluation 

ORDALT 

ordnance  alteration 

OSD 

Office  of  the  Secretary  of  Defense 

OTS 

off-the-shelf 

OUSD  (AT&L) 

Office  of  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics 

PA 

performance  assessment 

PARCA 

Performance  Assessments  and  Root  Cause  Analysis 

PARM 

participating  acquisition  resource  manager 

PAUC 

program  acquisition  unit  cost 

PDR 

Preliminary  Design  Review 

PE 

program  element 

PEO 

Program  Executive  Office 

PM 

program  manager 

PMA 

Program  Management  Activity 

PMS 

Program  Manager,  Ship 

PR 

program  review 

RAM 

rolling  airframe  missile 

RCA 

root  cause  analysis 

RDA 

research,  development,  and  acquisition 

RDT&E 

research,  development,  test,  and  evaluation 

RF 

radio  frequency 

RMP 

Risk  Management  Plan 

ROS 

reduced  operational  status 

SAC 

Senate  Appropriations  Committee 

SAE 

service  acquisition  executive 

SAR 

Selected  Acquisition  Report 

SASC 

Senate  Armed  Services  Committee 
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SDS 

system  design  specification 

SE 

systems  engineering 

SEP 

Systems  Engineering  Plan 

SFAE-AV-A9 

U.S.  Army  Program  Manager  for  Aviation  Systems 

SLAMRAAM 

surface-launched  medium-range  air-to-air  missile 

SLEP 

Service  Life  Extension  Program 

STRC 

Standards  in  Training  Commission 

SUAS 

small  unmanned  aircraft  system 

SYSCOM 

Systems  Command 

TEMP 

Test  and  Evaluation  Master  Plan 

TRA 

technology  readiness  area 

UAV 

unmanned  aerial  vehicle 

UAS 

unmanned  aerial  system 

UCR 

Unit  Cost  Report 

USA 

U.S.  Army 

USAF 

U.S.  Air  Force 

USN 

U.S.  Navy 

VCSA 

Vice  Chief  of  Staff  of  the  Army 

VTUAV 

vertical  takeoff  and  landing  tactical  unmanned  aerial  vehicle 

W&TCV 

weapons  and  tracked  combat  vehicles 

WGS 

wideband  global  satellite 

WSARA 

Weapon  Systems  Acquisition  Reform  Act 

XLWB 

extra-long  wheel  base 

CHAPTER  ONE 


Introduction 


In  addition  to  doing  root  cause  analyses  on  major  defense  acquisition  programs 
(MDAPs)  that  incurred  Nunn-McCurdy  breaches,  the  Director  of  the  Office  of  Per¬ 
formance  Assessments  and  Root  Cause  Analyses  (PARCA)  asked  RAND  to  explore 
some  additional  issues  to  determine  whether  they  might  affect  the  management  of  such 
programs.  This  report  presents  research  conducted  on  three  relevant  topics:  the  tenure 
of  program  managers,  the  need  for  Department  of  Defense  (DoD)— level  oversight  on 
Acquisition  Category  (ACAT)  II  programs,  and  the  potential  use  of  framing  assump¬ 
tions  as  a  way  to  manage  program  risk.  The  issues  raised  by  these  topics  are  discussed 
briefly  below. 


Tenure  Research 

In  November  2007,  the  U.S.  Government  Accountability  Office  (GAO)  released  a 
report  that  observed  the  following  regarding  program  manager  (PM)  tenure: 

while  DoD  policy  provides  for  program  managers  of  major  defense  acquisition 
programs  to  serve  as  close  to  4-year  tenures  as  practicable,  many  serve  for  only 
2  years.  .  .  .  our  work  has  shown  that  rather  than  lengthy  assignment  periods 
between  key  milestones  as  suggested  by  best  practices,  many  of  the  programs  we 
have  reviewed  had  multiple  program  managers  within  the  same  milestone.  .  .  . 
analysis  for  this  review  showed  that  for  39  major  acquisition  programs  started  since 
March  2001,  the  average  time  in  development  was  about  37  months.  The  average 
tenure  for  program  managers  on  those  programs  during  that  time  was  about  17.2 
months — less  than  half  of  what  is  prescribed  by  DoD  policy.1 

PM  tenure  has  long  been  a  topic  of  interest  to  DoD,  which  has  established  poli¬ 
cies  designed  to  increase  that  tenure,  most  recently  in  2005  and  2007.  The  Office  of  the 
Secretary  of  Defense  (OSD)  asked  RAND  to  conduct  research  on  the  tenure  of  PMs 


1  U.S.  Government  Accountability  Office,  Defense  Acquisitions:  Department  of  Defense  Actions  on  Program  Man- 
ager  Empowerment  and  Accountability,  GAO-08-62R,  Washington,  D.C.,  November  9,  2007a,  p.  8. 
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in  MDAPs.  The  policy  question  addressed  is  whether  PM  tenure  has  lengthened  since 
the  2005  and  2007  policy  guidance  that  attempted  to  extend  it.2  OSD  PARCA  is  also 
interested  in  any  historical  and  recent  outcomes  in  PM  tenure  in  MDAPs.  In  addition, 
it  is  seeking  an  alternative  to  GAO’s  2007  analysis  of  PM  tenure.  Reviewing  data  after 
2007  is  one  way  to  identify  whether  PM  tenure  policies  have  been  able  to  increase  the 
tenures  of  PMs  as  intended. 


ACAT  II  Programs 

Cost  growth  in  defense  acquisition  and  procurement  programs  has  long  been  an  issue 
of  concern  to  policymakers.  The  size  of  these  programs,  their  often  long  development 
time  lines  and  significant  costs,  and  the  need  for  repair  and  modernization  of  equip¬ 
ment  create  significant  risk  for  unexpected  costs  that  exceed  expectations  and  slips 
in  schedule  that  delay  the  delivery  of  important  systems.  DoD  also  often  depends  on 
contractors  to  produce  and  supply  procured  equipment  and  technology,  raising  con¬ 
cerns  about  contractor  incentives  and  competition  as  well  as  oversight  and  monitoring. 
There  have  been  many  legislative  attempts  to  control  cost  growth  and  development 
delays.  These  have  achieved  some  success  but  have  failed  to  either  fully  control  unex¬ 
pected  increases  in  program  costs  or  eliminate  long  production  delays.  In  addition, 
existing  requirements  cover  primarily  the  largest  acquisition  programs — MDAPs — 
leaving  smaller  and  non-MDAP  programs  without  consistent  oversight  at  the  OSD 
level.  Oversight  for  these  smaller  programs  occurs  only  at  the  military  department 
level.3  The  motivating  question  driving  this  report  is  whether  existing  decentralized 
systems  used  to  track  cost  growth  and  performance  of  ACAT  IIs  are  sufficient  or 
whether  additional  centralized  guidance  is  warranted. 


Framing  Assumptions 

Defense  acquisition  programs  routinely  face  the  challenge  of  estimating  cost,  sched¬ 
ule,  and  technological  performance  far  in  advance  of  actual  work.  They  must  also 
account  for  differences  in  acquisition  strategy  and  market  conditions.  As  a  result,  pro¬ 
grams  must  make  a  number  of  assumptions  about  their  programs  and  the  conditions 


2  The  guidance  is  presented  in  the  following  DoD  documents:  U.S.  Department  of  Defense  Instruction  (DoDI) 
5000.66,  Operation  of  the  Defense  Acquisition,  Technology,  and  Logistics  Workforce  Education,  Training,  and  Career 
Development  Program,  December  21,  2005;  and  Kenneth  J.  Krieg,  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics,  “Program  Management  Tenure  and  Accountability,”  memorandum,  Washington, 
D.C.,  May  25,  2007. 

3  Programs  are  designated  as  MDAPs  based  on  a  number  of  factors,  including  total  lifetime  cost. 
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that  might  affect  them.  Although  a  necessity,  assumptions  expose  the  program  to  cost 
growth  and  schedule  delays. 

When  assumptions  prove  faulty,  they  can  cause  the  program  to  miss  important 
cost  and  schedule  benchmarks.  More  important,  they  can  cause  programs  to  breach 
Nunn-McCurdy  thresholds,  which  can  lead  to  a  program’s  termination.  Key  assump¬ 
tions  are  called  “framing”  because  they  so  influence  the  program  performance  expec¬ 
tations  that,  when  faulty,  they  lead  to  significant  cost  growth,  performance  shortfalls, 
schedule  slips,  or  any  combination  of  these  three  undesirable  outcomes. 

This  exploratory  research  was  done  for  the  PARCA  office  in  an  attempt  to  define 
framing  assumptions  in  a  way  that  others  can  use  them  to  assess  and,  potentially,  con¬ 
trol  program  risk  and  to  explore  the  possibility  of  identifying  them  for  a  selected  set  of 
programs. 


Organization  of  This  Report 

This  report  is  organized  as  follows.  Chapter  Two  explores  the  topic  of  PM  tenure  over 
the  past  40  years.  The  chapter  specifically  looks  at  policies  and  recommendations, 
along  with  how  PM  tenure  has  been  quantified  in  the  literature.  It  next  provides  our 
analysis  of  PM  tenure  using  data  available  since  1997.  The  analysis  initially  focuses  on 
data  presented  in  the  Selected  Acquisition  Reports  (SARs)  for  PM  tenure  periods  in 
which  PMs  have  completed  their  tenure.  It  then  provides  a  short  statistical  analysis 
that  incorporates  more  recent  data  on  PMs  who  have  not  completed  their  tenure.  The 
goal  of  the  latter  analysis  was  to  reach  conclusions  regarding  PM  tenure  length  and  the 
2005  and  2007  policies  that  attempted  to  lengthen  tenure.  The  final  part  of  the  chapter 
presents  our  conclusions. 

Chapter  Three  deals  with  the  topic  of  oversight  for  ACAT II  programs.  It  describes 
the  existing  oversight  policies  and  the  practices  in  the  military  services.  It  then  com¬ 
pares  12  programs  (when  treating  Hellhre  as  a  separate  program),  MDAP  program 
performance  with  that  of  ACAT  II,  describes  the  factors  that  contribute  to  success  and 
failure,  and  lays  out  three  courses  of  action  that  the  PARCA  office  could  pursue  con¬ 
cerning  the  issuance  of  guidance. 

Chapter  Four  explores  the  topic  of  framing  assumptions  and  their  possible  use  as 
a  way  to  manage  risk  in  acquisition  programs.  It  begins  by  defining  framing  assump¬ 
tions  and  listing  their  salient  characteristics.  It  then  tests  the  feasibility  of  identifying 
framing  assumptions  by  examining  five  DoD  acquisition  programs.  It  also  takes  a 
retrospective  look  at  the  programs  for  which  the  RAND  National  Defense  Research 
Institute  has  done  root  cause  analyses.  It  concludes  by  providing  some  overarching 
observations. 

Chapter  Five  briefly  summarizes  conclusions  relating  to  the  three  issues  of  interest. 


CHAPTER  TWO 


Program  Manager  Tenure 


As  outlined  in  Chapter  One,  PM  tenure  has  long  been  a  topic  of  interest  to  DoD, 
which  has  established  policies  designed  to  increase  that  tenure,  most  recently  in  2005 
and  2007. 

Additional  exploration  of  other  research  quantifying  PM  tenure  since  2007  did 
not  yield  any  relevant  answers  to  the  research  question  of  whether  PM  tenure  has 
lengthened  as  a  result  of  these  policies.  However,  a  look  back  over  the  past  40  years 
revealed  that  PM  tenure  is  not  a  new  policy  issue,  and  it  has  been  measured  sev¬ 
eral  times  in  sample  sets  of  programs,  by  individual  service,  and  also  by  all  services 
collectively. 


Research  Approach 

PARCA  requested  that  RAND  calculate  current  PM  tenure  using  easily  available 
sources  to  determine  the  evidence  base  since  GAO’s  2007  assessment.  Part  of  the  moti¬ 
vation  behind  the  request  was  to  determine  whether  tenure  periods  have  been  increas¬ 
ing  since  the  2005  and  2007  guidance  as  a  way  to  better  understand  the  reasons  for 
program  success  or  failure.  The  RAND  team  completed  a  literature  review  to  identify 
previously  existing  PM  tenure  policies  and  to  track  how  other  studies  have  quantified 
PM  tenure. 

To  quantify  PM  tenure  using  current  data,  we  extracted  program  point-of- 
contact  data  from  the  SARs  for  1997-2011.  This  method  was  advantageous  because 
the  data  were  standardized  in  one  location  and  were  easily  extracted  from  OSD’s 
Defense  Acquisition  Management  Information  Retrieval  (DAMIR)  system.  In  addi¬ 
tion,  given  that  that  the  available  data  went  back  to  1997,  there  was  sufficient  informa¬ 
tion  on  a  variety  of  programs.  We  chose  to  use  data  from  MDAPs  that  had  multiple 
PMs,  because  we  could  define  the  tenure  of  a  particular  PM  as  the  date  that  the  PM 
was  assigned  to  the  date  that  the  subsequent  PM  was  assigned.  This  approach  provided 
distinct  starting  and  ending  points  for  each  PM  in  the  data  set.  For  PMs  with  open 
tenure  periods  (that  is,  the  period  of  service  by  a  program’s  current  PM),  we  conducted 
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a  statistical  analysis  to  determine  the  effects  of  open  tenure  periods,  particularly  in  the 
most  recent  years. 


Data  Shortcomings 

The  data  used  for  the  analysis  had  several  notable  shortcomings.  Some  PM  tenure  peri¬ 
ods  did  not  have  an  end  date,  either  because  the  PMs  were  still  with  the  program  or 
because  the  program  was  no  longer  submitting  SARs.  Finding  end  dates  for  programs 
when  this  information  was  not  included  in  the  data  would  have  required  a  significant 
amount  of  research  beyond  of  the  scope  of  this  study,  so  we  excluded  these  data.  There 
were  also  several  raw  data  errors  in  which  the  start  date  for  one  PM  was  announced  but 
the  following  chronological  SAR  listed  a  different  PM  with  a  start  date  before  that  of 
the  previous  PM.  This  caused  negative  numbers  in  tenure  periods.  Thus,  we  excluded 
these  data  also.  Some  PMs  managed  several  programs,  so  we  had  to  count  “tenure  peri¬ 
ods”  rather  than  PMs.  In  addition,  the  data  pulled  from  the  SARs  do  not  differentiate 
between  acting  and  permanent  PMs,  so  very  short  periods  may  indicate  changeover 
between  PMs,  in  which  the  deputy  PM  is  acting  in  the  PM  role  temporarily.  Shorter 
periods  may  also  indicate  pre-Milestone  (MS)  B  programs,  and  longer  periods  could 
reflect  data  reporting  issues  between  subprograms. 


The  Past  40  Years 

Over  the  past  40  years,  various  policies  have  been  implemented  in  an  attempt  to 
increase  PM  tenure  in  defense  programs.  Several  studies  have  also  attempted  to  quan¬ 
tify  the  tenure  of  PMs.  For  example,  GAO,  the  Defense  Science  Board  (DSB),  the 
RAND  Corporation,  and  the  Institute  for  Defense  Analyses  have  all  done  work  quan¬ 
tifying  PM  tenure. 

Policies  and  Recommendations 

Recommendations  and  policy  actions  have  appeared  in  a  variety  of  sources  since  the 
early  1970s.  Deputy  Secretary  of  Defense  David  Packard  addressed  PM  tenure  in  his 
speeches  and  memorandums  from  the  early  1970s.  According  to  Dews  et  ah,  one  major 
policy  element  of  Packard’s  initiatives  was  to  “reduce  the  turnover  rate  of  program 
managers  so  that  they  have  longer  job  tenure.”1  Likewise,  in  1973,  a  DSB  report  on 
reducing  the  costs  of  defense  system  acquisition  addressed  the  role  of  PM  tenure  by 
pointing  out  that  “the  program  manager’s  tenure  may  be  abbreviated  due  to  rotational 


1  Edmund  Dews,  Giles  K.  Smith,  Allen  Barbour,  Elwyn  Harris,  and  M.  Hesse,  Acquisition  Policy  Effectiveness: 
Department  of  Defense  Experience  in  the  1970s,  Santa  Monica,  Calif.:  RAND  Corporation,  R-2516-DR&E,  1979, 
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tours  of  duty  that  stress  command  responsibilities,  rather  than  management  achieve¬ 
ment,  for  recognition  and  advancement.”2  As  a  consequence,  the  DSB  recommended 
that  “the  tenure  of  key  DoD  program  people  be  increased,  at  least  to  coincide  with  the 
beginning  and  end  of  major  phases  of  a  program.”3  The  DSB  recommendation  was  fol¬ 
lowed  up  shortly  thereafter  by  DoD  Directive  (DoDD)  5000.23  in  November  1974, 
which  looked  at  “System  Acquisition  Management  Careers.”  Though  not  as  specific 
as  the  DSB  recommendation,  the  policy  stated  that  “the  tenure  of  program  manager 
assignments  should  be  sufficient  to  ensure  management  continuity.”4 

About  ten  years  after  the  1973  DSB  study,  the  board  surveyed  the  services  and 
contractors  regarding  selection,  training,  career  patterns,  and  the  length  of  assignment 
of  PMs.  It  concluded  that  “management  continuity  during  the  start-up  of  production 
was  critical  and  suggested  that  the  milestone  provision  be  modified”  based  on  data 
indicating  that  PM  turnover  is  much  more  likely  during  the  early  phases  of  a  program.5 
The  panel  also  concluded  that  “tour  lengths  for  military  program  managers  appear 
too  short,”6  adding  that  program  management  should  be  improved  by  “stabilizing  the 
tours  of  military  program  managers.  Extend  tours  beyond  the  current  practice  of  about 
30  months.”7 

The  Defense  Acquisition  Workforce  Improvement  Act  was  enacted  in  November 
1990.  Two  years  later,  in  1992,  GAO  reviewed  some  of  the  act’s  provisions  regarding 
PMs.  Particularly  applicable  to  our  study  were  the  findings  concerning  tenure  require¬ 
ments  for  PMs: 

Effective  October  1,  1991,  the  act  required  that  newly  appointed  program  manag¬ 
ers  of  major  and  significant  nonmajor  defense  acquisition  programs  .  .  .  agree  to  a 
tenure  requirement  that  they  remain  in  their  position  until  the  completion  of  the 
first  major  milestone  closest  in  time  to  the  date  they  had  served  4  years,  and  sign 
a  written  agreement  to  remain  on  active  duty  or  in  federal  service,  as  applicable, 
during  this  period.8 


2  Office  of  the  Director  of  Defense  Research  and  Engineering,  Defense  Science  Board  Task  Force,  Reducing 
Costs  of  Defense  Systems  Acquisition:  Design-to-Cost,  Commercial  Practice  vs.  Department  of  Defense  Practice,  Wash¬ 
ington,  D.C.:  Defense  Science  Board,  AD-762-487,  March  15,  1973,  p.  xvi. 

3  Office  of  the  Director  of  Defense  Research  and  Engineering,  Defense  Science  Board  Task  Force,  1973,  p.  29. 

4  U.S.  Department  of  Defense  Directive  5000.23,  System  Acquisition  Management  Careers,  November  26,  1974. 

5  Quoted  in  U.S.  General  Accounting  Office,  DoD  Acquisition:  Strengthening  Capabilities  of  Key  Personnel  in 
Systems  Acquisition,  Washington,  D.C.,  GAO/NSIAD-86-45,  May  1986a,  p.  75. 

6  Defense  Science  Board,  Report  of  the  Defense  Science  Board  Task  Force  on  Transition  of  Weapons  Systems  from 
Development  to  Production ,  Washington,  D.C.:  Office  of  the  Under  Secretary  of  Defense,  Research  and  Engineer¬ 
ing  August  1983,  p.  47. 

7  Defense  Science  Board,  1983,  p.  52. 

8  U.S.  General  Accounting  Office,  Defense  Management:  Implementation  of  the  Defense  Acquisition  Workforce 
Improvement  Act,  Washington,  D.C.,  GAO/NSIAD-92-97,  January  1992,  pp.  2—2. 
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The  above  tenure  agreements  were  in  place  for  nearly  15  years  before  Under  Sec¬ 
retary  of  Defense  for  Acquisition,  Technology,  and  Logistics  Kenneth  Krieg  reiter¬ 
ated  the  importance  of  acquisition  position  tenure  requirements  for  PMs  working  on 
MDAPs  and  major  automated  information  systems  (MAIS)  in  DoDI  5000.66.  The 
new  policy  stated  that  a  PM  should  have  a  tenure  period  around  the  “milestone  closest 
to  4  years  or  as  tailored  by  [Component  Acquisition  Executive]  based  on  unique  pro¬ 
gram  requirements.  [PMs  should  also]  execute  a  written  tenure  agreement.”9 

Following  this  guidance  was  Section  853  of  the  John  Warner  National  Defense 
Authorization  Act  for  Fiscal  Year  2007.10  The  law,  which  passed  in  September  2006, 
directed  “the  Department  [of  Defense]  to  revise  guidance  on  qualifications,  resources, 
responsibilities,  tenure,  and  accountability  of  program  managers  before  and  after  Mile¬ 
stone  B.”u  In  response  to  the  act,  Under  Secretary  Krieg  issued  a  memorandum  on 
May  25,  2007,  stating  that,  “in  accordance  with  10  U.S.C.  1734,  the  tenure  period 
for  program  managers  of  major  defense  acquisition  programs  shall  correspond  to  the 
major  milestone  closest  to  4  years,  subject  to  an  exceptional  circumstances  waiver.” 
Tenure  agreements  should  be  used  in  Acquisition  Category  I  and  II  programs.12 

In  2008,  Krieg’s  successor,  John  Young,  again  stressed  the  importance  of  PM  per¬ 
formance  and  of  having  a  strategy  in  place  to  ensure  longer  tenures.  He  testified  before 
Congress  regarding  the  initiatives  in  place  as  of  2008: 

I  have  put  in  place  a  comprehensive  strategy  to  address  improving  the  performance 
of  program  managers.  Key  to  this  are  program  manager  tenure  agreements  for 
Acquisition  Category  (ACAT)  I  and  II  programs,  which  are  our  largest  programs. 

My  expectation  is  that  tenure  agreements  should  correspond  to  a  major  milestone 
and  last  approximately  4  years.  Another  fundamental  piece  I  have  established  is 
Program  Management  Agreements — a  contract  between  the  program  manager 
and  the  acquisition  and  requirements/resource  officials — to  ensure  a  common 
basis  for  understanding  and  accountability;  that  plans  are  fully  resourced  and 
realistically  achievable;  and  that  effective  transparent  communication  takes  place 
throughout  the  acquisition  process.13 


9  U.S.  Department  of  Defense  Instruction  5000.66,  2005,  p.  21. 

10  Public  Law  109-364,  John  Warner  National  Defense  Authorization  Act  for  Fiscal  Year  2007,  October  17, 
2006. 

11  See  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics,  Report  to  Congress 
on  Program  Manager  Empowerment  and  Accountability,  Washington,  D.C.:  U.S.  Department  of  Defense,  August 
2007,  p.  1-2. 

12  Krieg,  2007,  p.  1. 

13  John  J.  Young,  Under  Secretary  of  Defense,  Acquisition,  Technology,  and  Logistics,  testimony  before  the  U.S. 
Senate  Committee  on  Armed  Services,  Washington,  D.C.,  June  3,  2008,  pp.  3-4 
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These  examples  of  policy  and  study  recommendations  illustrate  the  long-standing 
view  that  PM  tenure  should  be  actively  monitored  to  ensure  leadership  stability  in 
acquisition  programs.  The  repetition  of  themes  over  the  past  40  years  also  illustrates 
the  difficulty  of  implementing  and  enforcing  these  policies.  The  next  section  presents 
some  quantitative  data  from  the  same  period  to  provide  a  better  understanding  of  PM 
tenure  over  time. 

Quantifying  Program  Manager  Tenure  over  the  Past  40  Years 

In  October  1979,  RAND  published  a  report  that  attempted  to  quantify  PM  tenure 
using  data  from  the  military  services.  The  authors  found  that  PM  tenure  averaged  two 
and  a  half  to  three  years  and  that  there  had  been  an  increasing  trend  in  tenure  since 
the  mid-1960s: 

By  the  end  of  the  1960s  it  had  become  apparent  that  acquisition  programs  often 
suffered  from  too  rapid  a  turnover  of  program  managers.  Program  duration  some¬ 
times  exceeded  10  years,  and  program  managers  frequently  served  less  than  2  years 
on  the  job. ...  In  the  period  1961-1965,  for  example,  the  average  tenure  for  all  pro¬ 
gram  managers  was  18  months;  for  Army  program  managers,  the  average  was  only 
12  months.  .  .  .  The  data  on  program  manager  tenures  from  1961  to  1978  indicate 
a  steady  increase  in  tenure  from  an  all-Service  average  of  about  18  months  in  the  5 
years  centered  on  1963  to  about  32  months  in  the  5  years  centered  on  1976. . . .  The 
steady  upward  trend  in  the  5 -year  moving  averages  that  was  already  established  in 
the  1960s  has  continued  in  the  1970s.  Since  1969,  the  5-year  moving  average  has 
lengthened  from  26  months  to  32  months,  an  increase  of  nearly  one-fourth.  .  .  . 

But,  as  in  so  many  instances,  a  direct  causal  connection  between  these  elements  of 
policy  and  practice  cannot  be  established.14 

The  RAND  research  was  followed  by  a  DSB  study  in  1983  that  surveyed  the 
services  and  contractors  regarding  selection,  training,  career  patterns,  and  length  of 
assignment  among  PMs.  The  survey  results  indicated  that  the  average  tour  length  for 
military  PMs  had  improved  but  still  averaged  less  than  three  years.  The  study  also 
pointed  out  the  drastic  difference  in  PM  tenure  between  government  and  industry, 
stating  that  “industry  program  managers  have  tours  of  up  to  10  years  with  four  to  six 
years  being  quite  common.”15  The  panel  concluded  that  PM  tenure  was  too  short  and 
should  be  extended  “beyond  the  current  practice  of  about  30  months.”16 

In  the  mid-1980s,  GAO  also  released  a  report  addressing  PM  tenure,  and,  as  in 
the  1979  RAND  study,  tried  to  provide  some  quantitative  data.  GAO  used  case-study 
programs  but  also  tried  to  look  at  tenure  in  the  early  and  later  phases  of  programs: 


14  Dews  et  al.,  1979,  pp.  15-16. 

15  Defense  Science  Board,  1983,  p.  46. 

16  Defense  Science  Board,  1983,  p.  52. 


10  Management  Perspectives  on  Root  Cause  Analyses  of  Nunn-McCurdy  Breaches,  Volume  4 


Programs  experienced  considerable  turnover  in  program  managers  during  their 
earliest  phases,  particularly  in  the  Army  and  the  Air  Force.  In  our  17  case  study 
programs,  the  tenure  of  program  managers  who  had  been  replaced  averaged 
9  months  for  3  Army  programs,  16  months  for  6  Air  Force  programs,  and  39 
months  for  6  Navy  programs.  As  the  programs  progress,  tenure  tends  to  increase. 

Those  currently  serving  as  program  managers  have  been  in  their  positions  (as  of 
August  1986)  for  an  average  of  26  months  for  Army  programs,  31  months  for  Air 
Force  programs,  and  26  months  for  Navy  programs.  Tenure  of  program  managers 
replaced  from  January  1982  to  August  1984 — for  all  programs  and  phases — was 
longest  in  the  Navy.  The  average  tenure  of  Navy  program  managers  was  3.9  years, 
compared  with  3.1  years  for  Army  program  managers  and  only  1.9  years  for  Air 
Force  program  managers.17 

In  July  1986,  GAO  released  more  figures  on  program  manager  tenure.  At  that 
point,  GAO  reported,  “The  program  managers  averaged  approximately  27  months 
experience  on  their  current  program  as  either  the  program  manager  or  deputy  program 
manager.  The  deputies  had  approximately  30  months  of  experience  on  their  current 
program  as  either  the  deputy  or  the  program  manager.”18 

Continuing  the  trend  to  quantify  program  manager  tenure,  the  Institute  for 
Defense  Analyses  released  a  study  in  November  1988  stating  that,  in  the  Army  “the 
average  tenure  of  departing  program  managers  was  37.8  months  through  1987,”  and, 
in  the  Navy,  “about  23  months  at  the  end  of  1985.”  It  added,  “Tenure  at  departure  is 
not  reported.  However,  Navy  sources  say  that  most  program  managers  stay  for  almost 
a  full  four  years.”19 

In  October  2007,  the  Commission  on  Army  Acquisition  and  Program  Manage¬ 
ment  in  Expeditionary  Operations  used  a  prior  2001  Center  for  Naval  Analyses  study 
to  characterize  PM  tenure  in  the  Army.  The  study  rated  PM  tenure  as  “red”  in  both 
the  Army  and  Air  Force  but  “yellow”  in  the  Navy  and  Marine  Corps.  The  rationale  for 
the  Army  rating  was  that  “more  than  half  of  sampled  PMs  served  less  than  3  years.”20 

Some  of  the  most  recent  quantifications  of  PM  tenure  were  done  by  GAO.  In 
2005,  GAO  stated  that  PM  tenure  was  longer  but  that  most  PMs  still  left  after  three 


17  U.S.  General  Accounting  Office,  1986a,  pp.  79-80. 

18  U.S.  General  Accounting  Office,  Acquisition:  DoD’s  Defense  Acquisition  Improvement  Program:  A  Status 
Report,  Washington,  D.C.,  GAO/NSIAD-86-148,  July  1986b,  p.  11. 

19  David  R.  Graham,  Barbara  A.  Bicksler,  Robert  P.  Hilton,  Marshall  Hoyler,  Herschel  Kanter,  Walter  M. 
Locke,  George  H.  Sylvester,  and  John  R.  Transue,  Defense  Acquisition:  Observations  Two  Years  After  the  Packard 
Commission,  Institute  for  Defense  Analyses,  Volume  I:  Main  Report,  IDA  Report  R-347,  Arlington,  Va.:  Institute  for 
Defense  Analyses,  November  1988,  pp.  VI-4. 

20  Commission  on  Army  Acquisition  and  Program  Management  in  Expeditionary  Operations,  Urgent  Reform 
Required:  Army  Expeditionary  Contracting,  Washington,  D.C.,  October  31,  2007,  p.  18. 
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or  four  years.21  In  November  2007,  it  released  more  data  on  PM  tenure,  finding  signifi¬ 
cantly  shorter  averages.  In  this  second  study,  GAO  analyzed  39  MDAPs  that  started 
after  March  2001.  It  calculated  the  average  PM  tenure  for  these  programs  at  only  17.2 
months.22 

Table  2.1  summarizes  the  data  that  have  been  presented  on  program  manager 
tenure  in  the  literature  over  the  last  40  years.  These  data  indicate  that  average  PM 


Table  2.1 

Quantifying  PM  Tenure  over  the  Past  40  Years 


Average  PM  Tenure  (months) 


Approximate 
Years  of  Data 

All 

Services 

Air 

Force 

Army 

Navy 

Data  Source 

1961-1965 

18 

12 

Dews  et  al.,  1979,  pp.  5,  15 

End  of  1960s 

<  24 

Dews  et  al.,  1979,  pp.  5,  15 

Mid- 

1960s-1979 

30-36 

Dews  et  al.  1979,  pp.  5,  15 

1961-1978 

1961-1965:  18 
1974-1978:  32 

Dews  et  al.,  1979,  pp.  5,  15 

Circa  1983 

<36 

Defense  Science  Board,  1983,  p.  46 

January  1982- 
August  1984 

22.8 

37.2 

46.8 

U.S.  General  Accounting  Office,  1986a,  pp. 
79-80 

Circa  1986 

16 

9 

39 

U.S.  General  Accounting  Office,  1986a,  pp. 
79-80 

As  of  August 
1986 

31 

26 

26 

U.S.  General  Accounting  Office,  1986a,  pp. 
79-80 

Circa  1986 

27 

U.S.  General  Accounting  Office,  1986b,  p.  11 

1985-1987 

37.8 

23 

Graham  et  al.,  1988,  p.  VI-4 

Circa  2001 

<36 

Commission  on  Army  Acquisition  and 
Program  Management  in  Expeditionary 
Operations,  2007,  p.  18 

Circa  2005 

36-48 

U.S.  Government  Accountability  Office, 
2005,  p.  47 

2001-2007 

17.2 

U.S.  Government  Accountability  Office, 

2007,  p.  8 

21  U.S.  Government  Accountability  Office,  Best  Practices:  Better  Support  of  Weapon  System  Program  Managers 
Needed  to  Improve  Outcomes,  Washington,  D.C.,  GAO-06-110,  November  2005,  p.  47. 

22  U.S.  Government  Accountability  Office,  2005,  p.  8. 
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tenure  ranges  from  18  to  48  months.  However,  it  is  important  to  note  that  the  meth¬ 
odologies  vary  from  study  to  study,  which  reflects  the  difficulty  in  accurately  capturing 
how  long  PMs  have  remained  in  their  positions  over  time. 


Analysis  Using  Data  on  Recent  Program  Manager  Tenure 

The  wide  range  of  calculated  PM  tenures  reflects  several  data  sources  and  a  variety  of 
sample  sizes.  PARCA  requested  that  RAND  attempt  to  develop  a  separate  measure¬ 
ment  for  PM  tenure  using  easily  accessible  data.  As  explained  above,  we  used  data  from 
SARs  issued  between  1997  and  2011  to  quantify  PM  tenure;  this  is  a  standard  source 
with  easily  accessible  data  that  captures  the  largest  possible  sample  size  and  includes 
data  from  all  the  services.  The  remainder  of  this  chapter  presents  the  results  of  that 
analysis. 

RAND  Analysis  of  Data  on  Program  Manager  Tenure,  CY  1997-2011 

There  are  several  ways  to  present  the  results  of  our  analysis.  The  calculations  are  similar 
using  the  average  number  of  months,  but  our  sample  sizes  vary  based  on  whether  we 
decided  to  include  open  tenure  periods  and  outliers.  The  calendar  year  (CY)  1997- 
2011  SARs  included  PMs  who  started  their  tenure  as  far  back  as  1991.  Table  2.2  shows 
what  we  decided  to  exclude  in  the  three  ways  we  calculated  PM  tenure  (see  Figures  2.1, 
2.2,  and  2.3). 

The  first  sample  that  we  used  to  calculate  PM  tenure  excluded  open  tenure  periods 
but  included  PM  tenure  periods  that  started  as  early  as  1991,  along  with  other  outliers. 
Our  final  database,  excluding  incomplete  tenure  periods,  contained  370  PM  tenure 
periods  from  136  programs  that  had  submitted  SARs  (both  annual  and  quarterly) 
between  1997  and  2011.  Average  tenure  for  those  PM  periods  was  33.7  months,  which 
aligns  more  closely  with  the  2005  GAO  assessment  (no  more  than  three  to  four  years) 
than  its  2007  estimate  of  17.2  months.  Figure  2.1  shows  the  results  of  this  analysis. 


Table  2.2 

Exclusions  for  Quantifying  PM  Tenure 


Method 

Data  Range 
(years) 

Exclusions  from  Sample 

Figure  2.1 

1991-2011 

Excludes  open  tenure  periods  but 
includes  tenure  periods  that  started  as 
early  as  1991  as  well  as  other  outliers 

Figure  2.2 

1997-2008 

Excludes  the  outliers  by  eliminating  PM 
tenure  periods  starting  before  1997  and 
after  2008 

Figure  2.3 

1997-2008 

Excludes  data  before  1997  and  after 

2008  and  tenure  periods  greater  than  60 
months  and  less  than  7  months 
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Figure  2.1 

Average  Number  of  Months  PM  Is  Assigned  to  Program  (full  data  set) 


1990  1992  1994  1996  1998  2000  2002  2004  2006  2008  2010  2012 


Year  PM  assigned  to  program 

aRAND  calculated  average  of  370  program  manager  terms  using  1997  to  201 1  SAR  data. 
SOURCE:  SARs  (1997-2011). 

RAND  MC1 171 14-2.1 


As  the  figure  indicates,  the  spread  or  range  of  data  varies  considerably  across  years 
and  within  the  same  year.  There  are  also  several  outliers  with  short  terms  and  long 
terms,  as  well  as  a  few  data  points  before  1997  and  after  2008.  Figure  2.2  eliminates 
the  outliers  by  first  omitting  years  before  1997  and  after  2008.  After  eliminating  these 
outliers,  316  PM  tenure  periods  remained.  Given  this  sample,  the  total  average  tenure 
period  was  32.1  months.  This  does  not  differ  much  from  the  original  result  of  33.7 
months  but  is  still  significantly  higher  than  GAO’s  2007  estimate  of  17.2  months, 
which  had  a  sample  size  of  only  39  MDAPs. 

We  then  narrowed  the  data  set  even  further  by  eliminating  data  before  1997  and 
after  2008  and  by  disregarding  tenure  periods  greater  than  60  months  and  less  than 
seven  months.  We  did  this  to  remove  data  that  could  bias  the  results,  such  as  major 
outliers  and  tenure  periods  that  had  not  closed.  After  eliminating  these  data,  294  PM 
tenure  periods  remained.  This  third  sample  set  yielded  an  average  tenure  period  of  33.2 
months.  This  was  close  to  our  original  average  using  the  full  data  set,  but  it  was  still  sig¬ 
nificantly  different  from  GAO’s  2007  figure.  These  data  are  presented  in  Figure  2.3.  23 


23  We  began  the  analysis  with  a  full  data  set.  We  initially  found,  though,  that  the  data  were  “noisy.”  Accordingly, 
we  decided  to  run  scenarios:  full  data  set  and  excluding  what  we  thought  were  outliers  and  open  periods.  The 
point  of  this  exercise  was  to  see  if  there  was  any  major  bias  in  the  data  from  some  extreme  outliers  and  open  peri¬ 
ods.  A  careful  examination  of  the  composition  of  the  full  sample  set  led  us  to  conclude  that  the  outliers  would  be 
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Statistical  Analysis  including  Open  Tenure  Periods 

PARCA  was  specifically  interested  in  a  subset  of  our  overall  data  set  on  PM  tenure 
periods:  PM  tenure  periods  after  the  2005  and  2007  DoD  policy  guidance  that  both 
stressed  the  lengthening  of  these  periods.  Given  that  we  were  interested  in  data  near 
the  2005  and  2007  policy  actions,  we  also  needed  to  take  into  account  open  PM  tenure 
periods.  One  challenge  in  introducing  open  tenure  periods  to  this  analysis  was  that 
these  periods  cause  bias  in  the  average  or  mean,  because  the  PMs  have  not  finished 
their  tenure  in  a  particular  program.  This  makes  the  overall  average  shorter  than  if  the 
PMs  had  finished  their  tenure.  Understanding  that  this  issue  would  complicate  the 
analysis,  we  introduced  the  open  tenure  periods  that  we  had  previously  eliminated  and 
attempted  a  statistical  analysis  to  determine  whether  the  2005  and  2007  policies  were 
able  to  lengthen  tenure. 

Figure  2.2 

Average  Number  of  Months  PM  Is  Assigned  to  Program  (1997-2008  only) 
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SOURCE:  SARs  (1997-2008). 

NOTE:  Each  bar  represents  the  average  tenure  of  PMs  who  started  their  assignment  in  that  given  year. 
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under  seven  months  based  on  the  thought  that  these  PMs  were  most  likely  acting  PMs.  (Note:  The  SARS  have  no 
data  that  identify  acting  versus  permanent  PMs.)  The  reasons  for  the  upper  outliers  were  less  evident.  Nine  terms 
were  over  60  months.  We  have  some  interrelated  programs:  Ballistic  Missile  Defense  System  (BMDS)  and  Multi¬ 
functional  Information  Distribution  System  (MIDS);  Chemical  Demilitarization  (CHEM  DEMIL)  Assembled 
Chemical  Weapon  Alternatives  (ACWA)  and  Chemical  Materials  Agency  (CMA).  The  final  result  was  that  even 
when  we  took  out  the  outliers  (under  seven  months  and  over  60  months),  there  was  very  little  difference  in  the 
average  tenure:  33.7  months  compared  with  33.2. 
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Figure  2.3 

Average  Number  of  Months  PM  Is  Assigned  to  Program  (1997-2008  only,  excluding  tenure 
periods  of  more  than  60  months  and  less  than  7  months) 
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Including  open  tenure  periods  increased  our  data  set  to  464  total  tenure  periods. 
Of  these,  48  PMs  started  in  their  positions  before  1997,  and  416  started  between  1997 
and  2011.  Three  PMs  started  in  2011.  Obviously,  only  a  very  small  fraction  of  PM 
tenure  periods  that  started  in  2011  are  included  in  the  data.  Consequently,  in  this  data 
set,  we  excluded  entries  from  2011  and  before  1997. 

Table  2.3  shows  the  number  of  PM  tenure  periods  beginning  in  each  year.  The 
table  also  gives  the  percentage  of  these  positions  that  remained  open  at  the  time  of  the 
final  data  collection.  A  very  large  number  of  the  PMs  who  started  their  tenure  peri¬ 
ods  after  the  policy  change  were  still  in  their  positions  at  the  end  of  data  collection. 
This  makes  it  challenging  to  accurately  estimate  the  average  time  in  position  for  PMs 
following  the  policy  change.  Any  estimate  of  the  average  time  in  position  that  uses 
only  completed  tenures  will  be  severely  downwardly  biased  for  positions  opened  after 
2006,  because  it  is  likely  that  only  those  positions  with  short  tenures  will  be  available 
for  analysis.  For  example,  nearly  39  percent  of  PMs  who  started  in  their  positions  in 
2007  were  still  in  those  positions  when  we  collected  our  data.  The  average  tenure  of  the 
closed  positions  was  25  months.  The  average  tenure  of  open  positions  was  42  months  at 
the  time  of  data  collection,  which  means  that  the  final  tenure  must  average  at  least  this 
long  for  this  group.  The  average  tenure  for  all  cases  equals  0.61  times  the  average  tenure 
for  positions  that  had  closed  at  the  time  of  data  collection,  plus  0.39  times  the  average 
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Table  2.3 

PM  Tenure  Periods  and  Percentage  That  Remained 
Open  at  Last  Data  Collection,  by  Year 


Year 

Number  of  PM 
Tenure  Periods 

Percentage 

Open 

1997 

26 

0.0 

1998 

19 

0.0 

1999 

28 

0.0 

2000 

23 

0.0 

2001 

38 

0.0 

2002 

19 

0.0 

2003 

29 

0.0 

2004 

29 

0.0 

2005 

40 

0.0 

2006 

30 

3.3 

2007 

49 

38.8 

2008 

32 

68.8 

2009 

23 

87.0 

2010 

31 

96.8 

SOURCE:  SARs  (1997-2010). 


tenure  for  positions  that  remained  open  at  the  time  of  data  collection  (by  the  time  they 
eventually  close).  We  multiplied  the  mean  of  closed  positions  by  0.61  because  61  per¬ 
cent  of  positions  had  closed,  and  we  multiplied  the  mean  of  the  open  positions  by  0.39 
because  39  percent  of  the  positions  remained  open  at  the  time  of  data  collection.  The 
average  for  the  open  positions  is  unknown  but  must  be  greater  than  42  months,  so  the 
average  tenure  when  all  positions  close  will  be  greater  than  (0.61  x  25  months)  +  (0.39 
x  42  months)  =  32  months.  Consequently,  using  only  the  closed  positions  to  estimate 
average  tenure  would  bias  the  average  by  at  least  17  months  or  more  than  1.4  years.  The 
bias  would  be  even  larger  for  later  years,  when  even  more  positions  remained  open  at 
the  time  we  collected  our  data. 

Time  in  Position 

A  standard  solution  to  the  problem  of  studying  when  an  event  will  occur  before  it  has 
occurred  is  to  estimate  the  probability  that  the  event  will  occur  after  a  given  length  of 
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time.24  For  example,  we  could  estimate  the  probability  that  a  PM’s  tenure  will  be  longer 
than  one  day,  longer  than  two  days,  longer  than  three  days,  and  so  forth,  until  we  have 
covered  the  full  range  of  tenure  times.  The  probabilities  can  be  estimated  without  bias 
even  when  some  PMs  remain  in  their  positions.  Thus,  we  use  this  method  to  compare 
PM  tenures  in  three  distinct  periods:  before  the  first  policy  change  (positions  opened 
between  1997  and  2005),  after  the  first  policy  but  before  the  second  guidance  change 
and  GAO  report  (positions  opened  in  2006  and  2007),  and  after  the  policy  changes 
(positions  opened  in  2008  or  later). 

Figure  2.4  presents  the  results  of  our  analysis  in  graphical  form.  For  each  group 
of  programs — those  beginning  before  2006  (black  curve),  those  beginning  in  2006 
or  2007  (red  curve),  and  those  beginning  after  2007  (green  curve) — the  figure  plots 
the  probability  that  a  PM’s  tenure  was  a  given  number  of  days  or  longer.  As  the  figure 
shows,  no  positions  beginning  before  2007  remained  open  at  the  time  of  data  collec¬ 
tion,  whereas  large  numbers  of  PMs  remained  in  their  positions  among  those  starting 
in  2006  or  2007  or  after  2007. 

All  three  lines  are  at  1.00  for  days  zero  to  180,  because  we  excluded  programs 
in  which  PM  tenure  was  less  than  180  days.  Flence,  all  programs  in  the  sample  will 
last  180  days  or  longer.  After  180  days,  the  curves  turn  down  from  1.0  as  some  PMs’ 

Figure  2.4 

Distribution  of  Time  in  Position  Before  and  After  Policy  Change,  All  Periods 
(excluding  those  closed  in  less  than  180  days) 


- Positions  opened 

before  2006 

- Positions  opened 

after  2007 

- Positions  opened 

in  2006  or  2007 

+  Days  in  position 
that  remained 
open  at  the  time 
of  data  collection 


24  David  W.  Hosmere,  Jr.,  and  Stanley  Lemeshow,  Applied  Survival  Analysis:  Regression  Modeling  of  Time  to  Event 
Data,  New  York:  Wiley  Interscience,  1999. 
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tenure  ended  at  these  times.  At  one  year,  the  black  line  is  at  0.95,  indicating  that  about 
5  percent  of  PMs  beginning  their  positions  before  2006  had  left  those  positions  by  one 
year  later,  so  about  95  percent  would  remain  in  their  positions  for  a  year  or  longer.  By 
one  year,  nearly  11  percent  of  PMs  beginning  in  2006  or  2007  would  have  left  their 
positions,  so  about  89  percent  would  have  tenures  of  one  year  or  longer.  Only  about  3 
percent  of  PMs  starting  after  2007  left  their  programs  within  a  year,  so  97  percent  had 
tenures  of  more  than  one  year. 

For  the  first  two  years,  the  green  curve  is  higher  than  the  black  curve,  which,  in 
turn,  is  higher  than  the  red  curve,  indicating  that  PMs  starting  after  2007  had  the 
greatest  chance  of  remaining  in  their  positions  for  two  or  more  years,  and  PMs  start¬ 
ing  their  positions  in  2006  or  2007  had  the  lowest  chance  of  remaining  in  their  posi¬ 
tions  for  two  or  more  years.  The  green  curve  remains  above  the  other  two  curves  until 
around  day  950  (about  2.6  years). 

Positions  beginning  in  2006  or  2007  and  lasting  more  than  two  years  would  have 
been  closing  after  the  revised  guidelines  of  2007.  Combined  with  the  general  trend 
toward  longer  tenure  for  PMs  beginning  after  2007,  the  data  provide  some  support  for 
the  result  that  PMs  remained  in  their  positions  longer  after  the  release  of  the  revised 
guidelines  in  2007.  However,  because  of  the  small  sample  size  and  the  large  number  of 
open  positions  among  those  beginning  in  2006  or  later,  the  differences  among  groups 
are  not  statistically  significant,  and  the  observed  differences  could  be  chance  occur¬ 
rences  rather  than  real  differences  among  the  groups  of  positions. 

Additional  years  of  data  for  these  positions  and  new  positions  starting  in  2011  or 
later  could  help  establish  whether  the  differences  among  groups  demonstrate  persistent 
changes  in  practice  after  the  release  of  the  revised  guidelines.  Even  if  the  differences 
proved  to  be  persistent,  they  might  not  have  been  caused  by  the  revised  guidelines.  We 
would  need  to  rule  out  other  potential  sources  of  change  before  we  could  make  that 
attribution.  However,  analysis  of  additional  data  would  be  a  valuable  first  step  in  more 
fully  exploring  the  effects  of  the  2007  guideline  change. 


Conclusions 

PM  tenure  is  a  policy  issue  that  has  been  reviewed  and  quantified  periodically  over 
the  past  40  years.  It  is  difficult  to  assess  whether  policies  implemented  to  foster  longer 
tenures  have  been  successful  based  on  previous  data  and  studies  and  current  data.  It 
is  possible  to  conclude  that,  since  the  same  themes  have  reoccurred  in  policies  over 
the  past  40  years,  the  intent  of  these  policies  may  not  have  been  achieved.  Further¬ 
more,  an  enforcement  mechanism  has  not  been  readily  apparent  over  time.  This  could 
be  because  enforcement  is  limited  as  a  result  of  the  fundamental  conflict  that  exists 
between  what  military  officers  need  to  do  to  be  promoted  and  their  tenure  as  PMs. 
Unless  these  two  objectives  are  connected  so  that  lengthy  tenure  in  a  program  can  be 
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advantageous  for  promotion,  it  is  unlikely  that  these  tenure  policies  will  consistently 
yield  positive  results.  In  addition,  PM  tenure  can  be  affected  by  a  variety  of  issues  that 
we  have  not  taken  into  account  in  this  study.  These  include,  for  example,  the  following: 

•  moving  a  PM  from  one  program  to  another  prematurely  to  fulfill  a  gap  in  a  criti¬ 
cal  program 

•  premature  termination  of  a  program  and  consequent  premature  conclusion  of  a 
PM’s  tenure  period 

•  poor  performance  requiring  that  a  PM  be  moved  out  of  a  program 

•  ending  a  tenure  period  based  on  promotion  to  a  position  outside  the  program 
office. 

Given  the  narrow  focus  of  this  study,  we  did  not  take  into  account  other  data 
sets  that  might  have  been  relevant.  Expanding  the  focus  to  include  data  on  the  tenure 
agreements  between  PMs  and  acquisition  executives,  along  with  the  Precepts  for  the 
Promotion  Boards  in  the  Navy  and  similar  processes  in  the  other  services,  would  pro¬ 
vide  a  richer  analysis  of  this  topic. 

In  this  study,  we  attempted  to  quantify  PM  tenure  from  1997  through  2011  using 
SAR  data.  We  then  tried  to  determine  whether  the  policies  of  2005  and  2007  have 
helped  lengthen  PM  tenure.  A  statistical  analysis  using  open  and  closed  tenure  peri¬ 
ods  indicated  that  the  estimate  of  the  mean  time  in  position  for  tenure  periods  start¬ 
ing  after  2007  is  biased,  because  of  the  large  number  of  tenure  periods  that  remained 
open  at  the  last  data  collection.  This  creates  an  average  for  tenure  periods  starting  after 
2007  that  is  lower  than  will  be  the  case  when  all  PM  tenure  periods  starting  in  2008, 
2009,  and  2010  have  both  a  beginning  and  an  end.  Using  these  same  data,  we  found 
that  PMs  who  started  in  2006  or  2007  had  about  the  same  probability  of  remaining  in 
their  position  for  at  least  two  years  (0.75)  but  a  greater  probability  of  staying  for  three 
or  more  years  (0.64  compared  with  0.41)  than  PMs  who  started  in  2005  or  earlier. 
PMs  starting  in  2008  or  later,  had  about  an  87  percent  chance  of  remaining  in  their 
positions  for  two  or  more  years  and  about  a  50  percent  chance  of  remaining  for  three 
or  more  years,  suggesting  an  increase  in  PM  tenure  after  the  second  policy  change. 
The  differences  are  not  statistically  significant,  because  of  the  small  number  of  tenure 
periods  starting  after  2007  and  the  large  proportion  of  PMs  from  this  sample  who 
remained  in  their  positions  when  we  collected  the  data. 

We  also  found  that  adjusting  for  a  variety  of  data  issues  (e.g.,  outliers,  open  peri¬ 
ods)  gave  us  more  confidence  in  the  data  but  only  minimally  changed  the  result  of 
our  analysis  of  PM  tenure  in  MDAPs  over  the  past  14  years.  By  taking  into  account 
closed  PM  tenure  periods,  we  found  that  PM  tenure  averages  33  months.  This  result 
is  much  higher  than  GAO’s  2007  figure  of  17.2  months  (based  on  a  sample  size  of 
39  programs).  However,  our  figure  reflects  a  larger  sample  size  covering  more  years 
and  does  not  include  any  open  periods.  In  conclusion,  our  analysis  was  able  to  quan- 
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tify  PM  tenure  using  current  data  but  could  not  definitely  say  whether  recent  policies 
regarding  PM  tenure  have  had  any  positive  effect  on  lengthening  tenure  over  the  past 
several  years,  because  there  were  still  too  many  open  tenure  periods  at  the  time  of  data 
collection. 


CHAPTER  THREE 


Oversight  of  ACAT  II  Programs 


Do  ACAT  II  Programs  Need  Additional  Oversight? 

Many  legislative  attempts  have  been  made  to  control  cost  growth  and  to  eliminate 
development  delays.  These  have  achieved  some  success  but  have  failed  to  eliminate 
either  unexpected  increases  in  program  costs  or  long  production  delays.  In  addition, 
existing  requirements  cover  primarily  the  largest  acquisition  programs,  MDAPs,  leav¬ 
ing  smaller  and  non-MDAP  programs  without  consistent  centralized  oversight  at  the 
OSD  level.  Oversight  for  these  smaller  programs  occurs  only  at  the  military  depart¬ 
ment  level.1  The  motivating  question  driving  this  report  is  whether  existing  decentral¬ 
ized  systems  used  to  track  cost  growth  and  performance  of  ACAT  II  programs  are 
sufficient  or  whether  additional  centralized  guidance  is  warranted. 

Additional  monitoring  increases  the  burden  on  the  defense  agencies  and  military 
departments,  which  must  hie  extra  reports,  and  the  Office  of  the  Under  Secretary  of 
Defense  for  Acquisition,  Technology,  and  Logistics  (OUSD  [AT&T]),  which  will  be 
responsible  for  executing  new  control  actions.  As  a  result,  a  decision  to  mandate  addi¬ 
tional  reporting  and  control  requirements  for  ACAT  II  programs  is  warranted  only  if 
problems  with  the  existing  system  result  in  cost  growth  and  schedule  slippage  and  can 
be  reduced  or  eliminated  by  centralized  oversight  that  is  cost  effective. 

To  assess  if  such  performance  issues  do  plague  current  ACAT  II  programs,  we 
conduct  two  sets  of  analyses,  one  on  a  set  of  case  studies  of  ACAT  II  programs  and 
one  on  a  comparable  set  of  MDAPs.  Both  sets  include  programs  from  across  military 
departments  and  procurement  program  categories  (e.g.,  aircraft,  weapons,  shipbuild¬ 
ing,  and  conversions).  We  evaluate  the  performance  of  each  program  in  both  sets, 
focusing  specifically  on  unit  cost  growth  over  program  life  and  the  amount  of  insta¬ 
bility  in  annual  quantity  procured  over  time.  Then,  we  compare  the  overall  perfor¬ 
mance  of  the  two  sets  of  programs.  This  analysis  and  comparison  suggest  some  general 
inferences  about  the  performance  of  ACAT  II  programs  under  the  current  system  of 
oversight,  their  performance  relative  to  MDAPs,  which  are  subject  to  more  centralized 


1  Programs  are  designated  as  MDAPs  based  on  a  number  of  factors,  including  total  lifetime  cost.  See  Table  3.1 
for  details  and  a  complete  set  of  definitions. 
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monitoring  and  requirements,  and  the  need  for  additional  centralized  reporting  and 
control  requirements.  If  sample  ACAT  II  programs  perform  as  well  or  better  under 
the  current  decentralized  system  than  sample  MDAP  programs  under  more  rigorous 
oversight,  then  there  may  be  little  need  for  additional  reporting  and  control  require¬ 
ments  for  ACAT  II  programs.  If  sample  ACAT  II  programs  perform  worse  than  sample 
MDAPs,  additional  ACAT  II  oversight  may  be  warranted. 

A  few  words  about  the  scope  of  our  research  questions  and  analytical  approach 
provide  some  additional  context  for  the  reader.  First,  we  adopted  a  very  narrow  research 
question,  focused  explicitly  on  determining  whether  active  ACAT  II  programs  appear 
to  suffer  from  cost  and  schedule  problems  that  would  demand  additional  guidance. 
Although  identifying  the  primary  root  causes  and  measuring  the  precise  magnitude 
and  duration  of  performance  problems  in  ACAT  II  programs  are  related  and  impor¬ 
tant  questions,  they  fall  outside  the  scope  of  this  study.  Second,  we  used  only  publicly 
available  data  for  the  assessment  of  both  ACAT  II  and  MDAP  programs,  specifically, 
data  from  the  budget  justifications  written  by  the  military  departments.  We  inten¬ 
tionally  did  not  try  to  obtain  or  use  other  data  sources,  proprietary  information,  or 
personal  observations  or  opinion  held  by  program  managers  or  acquisition  executives 
within  the  military  departments.  Our  reliance  on  objective  and  consistent  data  sources 
for  all  programs  across  military  departments  supports  more  meaningful  comparison 
and  limits  concerns  of  bias  introduced  by  personal  reports.  As  important,  the  approach 
allows  us  to  investigate  the  quality  and  quantity  of  publicly  available  data  on  ACAT  II 
programs  and  to  define  a  methodology  for  conducting  research  on  and  evaluations  of 
these  smaller  acquisition  programs  in  the  future. 

Reliance  on  publicly  available  data  had  advantages  for  this  study,  but  it  also  had 
some  drawbacks.  For  example,  additional  data  or  explanations  directly  from  a  program 
manager  might  have  provided  more  nuanced  assessments  of  program  performance  or 
better  justifications  for  certain  cost  overruns  or  slips  in  schedule.  It  might  also  have 
highlighted  areas  where  acquisition  executives  feel  that  additional  oversight  might  be 
helpful  or  where  ACAT  II  programs  often  struggle  in  performance.  As  noted  above, 
we  chose  not  to  pursue  these  avenues,  because  our  limited  objective  was  to  conduct  a 
broad  comparison  of  ACAT  II  and  MDAP  performance  as  a  diagnostic  tool  to  assess 
the  need  for  additional  guidance. 

A  third  issue  related  to  the  scope  of  the  study  is  generalizability.  Since  we  assess 
only  a  selected  set  of  ACAT  II  and  MDAP  programs,  our  analysis  and  observations 
are  most  relevant  to  these  specific  programs.  However,  because  we  selected  our  sets  to 
include  a  range  of  different  types  of  programs  from  across  procurement  program  cat¬ 
egories  and  military  departments,  our  results  should  have  some  applicability  to  acquisi¬ 
tion  programs  more  generally. 

Finally,  our  assessments  of  cost  and  schedule  do  not  consider  research,  develop¬ 
ment,  test,  and  evaluation  (RDT&E)  costs,  which  often  account  for  a  large  portion  of 
cost  growth.  The  research  and  development  phase  is  also  often  associated  with  sched- 
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ule  delays.  The  decision  not  to  include  these  particular  costs  was  largely  driven  by  data 
availability:  Reliable  and  complete  RDT&E  costs  are  not  available  for  most  ACAT  II 
programs.  This  choice  means  that  our  analysis  will  not  capture  performance  issues  that 
affect  programs  before  their  entrance  into  production  (and  so  may  be  overly  positive) 
and  will  completely  miss  programs  that  never  make  it  out  of  the  development  stage. 
However,  these  limitations  are  largely  outside  the  very  narrow  focus  and  objective  of 
this  study,  which  is  to  determine  whether  active  ACAT  II  programs  need  additional 
oversight  to  limit  cost  growth.  For  this  purpose,  reliance  on  performance  data  from 
after  the  program  enters  production  should  be  sufficient.  Finally,  to  ensure  compara¬ 
bility  between  our  assessment  of  ACAT  II  and  MDAP  programs,  we  also  focus  on 
MDAP  performance  only  after  the  program  enters  production. 

The  considerations  above  do  not  reduce  the  relevance  or  value  of  the  analysis  pre¬ 
sented  in  the  remainder  of  the  report,  either  as  a  demonstration  of  a  methodological 
approach  to  studying  ACAT  II  program  performance  or  as  a  diagnostic  tool  used  to 
assess  the  potential  need  for  and  value  of  additional  guidance  for  ACAT  II  programs. 

Before  conducting  our  assessment,  however,  it  is  useful  to  consider  the  oversight 
mechanisms  and  reporting  and  control  requirements  that  existing  statute  defines  for 
acquisition  programs  as  well  as  the  current  procedures  used  by  the  military  depart¬ 
ments  to  monitor  the  cost  growth  and  overall  performance  of  these  programs.  The 
remainder  of  the  chapter  provides  some  detail  on  both  levels  of  performance  oversight. 

As  noted  in  the  introduction,  oversight  for  these  smaller  programs  occurs  only  at 
the  military  department  level.  Although  this  gap  in  oversight  may  initially  seem  wor¬ 
risome,  in  reality,  additional  centralized  guidance  is  valuable  only  when  it  is  needed 
or  provides  important  transparency  about  program  performance  that  improves  pro¬ 
gram  health  over  the  long  run.  In  fact,  additional  monitoring  comes  with  a  number  of 
drawbacks,  including  extra  burden  on  the  defense  agencies  and  military  departments, 
which  must  file  extra  reports,  and  OUSD  (AT&F),  which  will  be  responsible  for  exe¬ 
cuting  new  control  actions.  As  a  result,  a  decision  to  mandate  additional  reporting  and 
control  requirements  for  ACAT  II  programs  is  warranted  only  if  there  are  problems 
with  the  existing  system  that  result  in  cost  growth  and  schedule  slippage  and  that  can 
be  reduced  or  improved  by  centralized  oversight. 

In  this  chapter,  we  assess  whether  such  performance  issues  do  plague  current 
ACAT  II  programs  by  comparing  the  performance  of  a  set  of  ACAT  II  programs  with 
that  of  a  selected  set  of  MDAPs,  chosen  to  be  broadly  comparable  and  generalizable. 
We  use  this  analysis  and  comparison  to  evaluate  the  need  for  additional  oversight 
of  ACAT  II  programs.  Specifically,  additional  oversight  may  be  needed  only  if  the 
selected  set  of  ACAT  II  programs  perform  worse  than  selected  set  of  MDAPs.  As  noted 
above,  we  rely  on  publicly  available  data  and  keep  the  scope  of  our  research  narrow, 
explicitly  considering  only  whether  active  ACAT  II  programs  appear  to  suffer  from 
cost  and  schedule  problems  that  might  demand  additional  guidance.  We  also  consider 
programs  only  after  they  enter  production  and  we  exclude  RDT&E  costs. 


24  Management  Perspectives  on  Root  Cause  Analyses  of  Nunn-McCurdy  Breaches,  Volume  4 


Before  conducting  this  assessment,  however,  it  is  useful  to  consider  the  oversight 
mechanisms  and  reporting  and  control  requirements  that  existing  statute  defines  for 
acquisition  programs,  particularly  MDAP  programs,  as  well  as  the  current  procedures 
used  by  the  military  departments  to  monitor  cost  growth  and  the  overall  performance 
of  these  programs. 


Background  on  Existing  Oversight  for  MDAPs 

Once  an  acquisition  program  is  designated  an  MDAP,  it  receives  considerable  oversight 
to  ensure  that  it  meets  it  cost,  schedule,  and  technical  performance  requirements.  Three 
reports  are  designed  to  provide  the  information  necessary  to  carry  out  this  oversight: 
Selected  Acquisition  Reports,  Unit  Cost  Reports  (UCRs),  and  Defense  Acquisition 
Executive  Summaries  (DAES).  The  SARs  and  the  UCRs  are  statutory  requirements, 
and  the  DoD  regulation  requires  the  DAES.  In  addition  to  these  three  reports,  Con¬ 
gress  also  requires  that  DoD  establish  a  baseline  description  for  each  MDAP,  including 
cost,  schedule  and  performance  goals,  and  anything  else  deemed  pertinent.  The  Secre¬ 
tary  of  Defense  must,  by  law,  approve  cost,  schedule,  and  performance  goals  for  these 
programs.  Program  managers  must  document  goals  for  each  program  before  it  begins.2 

Nunn-McCurdy  Provisions  and  Breaches 

One  of  the  first  legislative  attempts  to  control  cost  growth  in  acquisition  and  procure¬ 
ment  programs,  the  Nunn-McCurdy  provisions,  was  initiated  by  Senator  Sam  Nunn 
and  Representative  David  McCurdy  as  part  of  the  1982  National  Defense  Authori¬ 
zation  Act.  The  Nunn-McCurdy  provisions,  still  in  use  today  in  an  updated  form, 
created  a  codified  way  for  Congress  to  keep  pervasive  cost  growth  in  check  through 
a  system  of  reporting  and  control  requirements.  The  provisions  use  unit  cost  growth 
as  one  metric  to  monitor  program  health.  The  two  unit  cost  definitions  are  program 
acquisition  unit  cost  (PAUC)  and  average  procurement  unit  cost  (APUC).  PAUC  is 
the  total  development,  procurement,  and  construction  cost  divided  by  the  total  pro¬ 
gram  quantity,  and  the  APUC  is  just  the  total  procurement  amount  divided  by  the 
procurement  quantity.  If  either  of  these  costs  exceeds  a  threshold  value,  then  the  pro¬ 
gram  is  said  to  incur  a  Nunn-McCurdy  breach,  which  triggers  automatic  procedures 
intended  to  mitigate  cost  growth.  Although  the  provisions  provided  some  transparency 
on  acquisition  programs  and  their  overall  health,  they  applied  only  to  the  largest,  those 
qualifying  as  MDAPs  based  on  their  expected  total  cost.3 


2  This  description  has  been  drawn  from  Gerry  Land,  Oversight  of  Major  Acquisition  Programs,  Fort  Belvoir,  Va.: 
Defense  Acquisition  University,  April  2006. 

3  10  U.S.  Code  Section  2433  Unit  Cost  Reports,  Washington,  D.C.:  Government  Printing  Office,  2010. 


Oversight  of  ACAT  il  Programs  25 


Modifications  to  the  law  in  2006  added  some  nuance  by  defining  significant  and 
critical  breaches.  A  significant  breach  occurs  when  either  PAUC  or  APUC  exceeds 
15  percent  of  the  current  baseline  estimate  or  30  percent  of  the  original  baseline  esti¬ 
mate.  A  critical  breach  is  for  a  25  or  50  percent  increase,  respectively.  The  comparison 
to  the  original  baseline  is  included  to  prevent  cost  growth  being  hidden  by  constant 
program  re-baselining. 

Changes  Brought  by  WSARA  2009 

The  original  Nunn-McCurdy  legislation  had  some  success  in  monitoring  and  limiting 
cost  growth  and  schedule  delays  by  illuminating  important  problems  and  program 
issues  but  was  not  able  to  eliminate  either  problem,  and  both  remained  significant 
impediments  to  acquisition  program  efficacy.  More  recent  legislation  has  attempted  to 
address  cost  growth  and  schedule  slippage  with  other  types  of  oversight  and  changes 
to  the  acquisition  processes  and  procedures.  One  comprehensive  effort,  the  Weapon 
Systems  Acquisition  Reform  Act  (WSARA)  of  2009,  sought  to  systematically  alter 
the  acquisition  process  through  changes  to  acquisition  organization,  personnel  struc¬ 
ture,  policy,  and  reporting  and  control  requirements.  Based  on  numerous  reports  and 
investigations  into  cost  growth  and  schedule  slippages  that  identified  the  lack  of  a  solid 
foundation  of  systems  engineering,  cost  estimation,  and  developmental  testing  early  in 
the  program  as  drivers  of  cost  growth  and  schedule  slippage,  WSARA  2009  laid  out 
specific  modifications  to  the  procurement  process  intended  to  bolster  these  aspects  of 
acquisition.4 

These  changes  can  broadly  be  categorized  into  three  areas:  organizational  changes, 
policy  changes,  and  reporting  and  control  requirements.5 

Organizational  Changes 

WSARA  2009  dictates  that  DoD  must  evaluate  its  systems  engineering  capabilities 
and  develop  skilled  employees  to  fill  any  gaps  in  their  ability  to  support  key  acqui¬ 
sition  decision  with  rigorous  systems  analysis  and  systems  engineering.  DoD  must 
also  do  the  same  for  developmental  testing  and  establish  a  new  position:  Director  of 
Developmental  Test  and  Evaluation.  The  Director  of  Defense  Research  and  Engineer¬ 
ing  (DDR&E),  an  extant  position,  would  have  new  responsibilities,  including  peri¬ 
odic  reviews  of  the  critical  technologies  used  in  MDAPs.  To  address  the  common 
problem  of  unrealistic  cost  estimates,  a  Director  of  Independent  Cost  Assessment 
would  be  established  to  ensure  that  the  cost  estimates  for  MDAPs  are  reasonable  and 
dependable.  In  addition,  to  better  solicit  feedback  from  the  end  user  of  MDAPs  (the 
warfighter),  the  Joint  Requirements  Oversight  Council  (JROC)  would  be  required  to 


4  Public  Law  111-23,  2009. 

5  Sen.  Carl  Levin,  Summary  of  the  Weapon  Systems  Acquisition  Reform  Act  of 2009,  press  release,  February  24, 
2009. 
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seek  and  consider  input  from  the  combatant  commanders  (COCOMs)  when  identi¬ 
fying  joint  military  requirements.  Finally,  two  new  positions  were  created  to  evaluate 
MDAPs:  the  Director  of  Cost  Assessment  and  Program  Evaluation  (CAPE)  and  the 
senior  official  for  PARCA. 

Policy  Changes 

WSARA  2009  requires  increased  interaction  between  the  traditionally  stovepiped 
requirements,  budget,  and  acquisition  aspects  of  procurement.  Because  cost,  schedule, 
and  performance  are  highly  intertwined,  this  increased  coordination  can  mitigate  the 
spread  of  problems  in  one  area  of  a  program  to  others.  To  ensure  a  solid  knowledge  base 
in  each  MDAP,  a  Preliminary  Design  Review  (PDR)  and  formal  post-PDR  assessment 
must  be  completed  before  a  MS  B  decision  can  be  made,  sending  the  program  into 
the  engineering,  manufacturing,  and  development  (EMD)  phase.  This  change  is  con¬ 
sistent  with  the  additional  responsibilities  of  the  DDR&E.  To  counteract  the  effects 
of  the  consolidation  of  the  defense  industry,  a  number  of  new  measures  promoting 
competitions,  as  well  as  periodic  program  reviews,  and  additional  competition  at  the 
subcontract  level  were  introduced.  These  measures  include  competitive  prototyping, 
dual-sourcing,  funding  of  a  second  source  for  next-generation  technology,  use  of  open 
architectures  to  ensure  competition  for  upgrades,  periodic  competitions  for  subsystem 
upgrades,  licensing  of  additional  suppliers,  and  government  oversight  of  make-or-buy 
decisions.  Additional  oversight  was  also  added  to  keep  organizational  conflict  of  inter¬ 
est  in  check. 

WSARA  also  changed  reporting  and  control  requirements  and  review  processes. 
First,  both  MS  A  and  MS  B  certification  processes  were  modified.  Under  new  guide¬ 
lines,  before  MS  A  certification,  each  program  must  undergo  a  Nunn-McCurdy-like 
review.  In  addition,  any  program  with  MS  B  certification  waived  must  undergo  an 
annual  review  by  the  Milestone  Decision  Authority  (MDA).  These  amendments  are 
applied  retroactively,  such  that  programs  with  MS  A  certification  and  no  MS  B  cer¬ 
tification  and  programs  with  MS  B  certification  and  no  MS  C  certification  must  be 
retroactively  recertified  using  the  new  requirements.  Second,  a  program  experiencing  a 
critical  Nunn-McCurdy  breach  must  have  a  root  cause  analysis  (RCA)  performed,  and 
the  program  may  be  terminated  unless  the  Secretary  of  Defense  certifies  to  Congress 
that  the  program  is  essential  to  national  security  and  can  be  modified  to  continue  in 
a  financially  sensible  manner.  The  program’s  most  recent  milestone  will  be  rescinded 
and  a  new  milestone  approval  must  be  obtained  before  any  new  contracts  or  contract 
modifications  can  be  signed.  These  revisions  were  intended  to  make  oversight  processes 
more  rigorous  and  direct  and  to  prevent  programs  with  significant  problems  from  elud¬ 
ing  management  oversight.  Once  again,  however,  their  reach  was  limited:  WSARA 
requirements  apply  only  the  largest  defense  acquisition  programs,  MDAPs,  and  not  to 
smaller  acquisition  programs. 
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Congressional  Reporting  and  Control  Requirements 

WSARA  2009  also  adds  reporting  and  control  requirements  that  communicate  prog¬ 
ress  on  acquisition  performance  to  Congress. 

•  The  director  of  CAPE  is  required  to  provide  an  annual  report  evaluating  the  pre¬ 
vious  year’s  cost  estimates.  The  CAPE  office  also  must  produce  a  one-time  report 
on  findings  and  recommendations  for  establishing  MDAP  operating  and  support 
cost  baselines. 

•  The  directors  of  developmental  test  and  evaluation  (DT&E)  and  systems  engi¬ 
neering  (SE)  must  produce  an  annual  joint  report  on  specific  MDAP-related 
activities. 

•  The  senior  official  for  PARCA  must  produce  an  annual  activities  report. 

•  The  director  of  research  and  engineering  (R&E)  must  provide  annual  assessments 
on  the  technological  maturity  of  technologies  critical  to  MDAPs. 

•  Finally,  more  earned  value  management  information  was  required  in  its  congres¬ 
sional  report. 


Implications  of  WSARA  for  Acquisition  Programs 

WSARA  is  intended  to  improve  the  acquisition  of  MDAPs,  but  many  of  the  provi¬ 
sions  carry  additional  costs  for  the  defense  agencies,  military  departments,  and  offices 
within  OUSD  (AT&L).  For  the  defense  agencies  and  military  departments,  the  most 
significant  burden  will  be  the  increase  in  workload  necessary  to  meet  all  new  reporting 
and  control  requirements.  More  independent  cost  estimates  are  necessary,  as  well  as 
the  ability  to  monitor  costs  continuously.  Annual  reports  must  be  presented  to  Con¬ 
gress,  all  requiring  additional  staffing  and  manpower  to  complete.  For  offices  within 
OUSD  (AT&L),  including  PARCA,  additional  centralized  oversight  will  also  increase 
the  staffing  burden  to  support  the  review  of  new  data  and  the  implementation  of  new 
control  mechanisms. 

WSARA  also  introduces  the  requirement  for  an  RCA  to  take  place  for  every 
MDAP  that  undergoes  a  Nunn-McCurdy  breach.  These  analyses  are  not  trivial  pursuits 
and  require  considerable  resources  and  expertise.  The  short  turnaround  time  required 
to  present  the  analysis  to  Congress  adds  another  layer  of  burden.  In  addition,  the  retro¬ 
active  recertifications  of  programs  between  Milestones  A  and  B  and  Milestones  B  and 
C  are  an  additional  strain  on  DoD.  To  fill  these  new  requirements,  existing  workforces 
must  be  retooled,  new  talent  recruited  and  hired,  and  organizations  restructured,  all  in 
a  period  of  constrained  resources.6 


6 


Mona  Lush,  Implementation  of  Weapon  Systems  Acquisition  Reform  Act  (WSARA)  of 2009,  October  22,  2009. 
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Oversight  of  Non-MDAPS 

As  noted  above,  existing  reporting  and  control  requirements  apply  only  to  MDAPs, 
the  largest  acquisition  programs  based  on  expected  lifetime  cost.  Many  other  classes  of 
acquisition  programs  are  defined  based  on  the  size  of  the  program  and  its  total  lifetime 
cost.  Table  3.1  summarizes  acquisition  program  types  and  definitions. 

Table  3.1 
Program  Types 


MDAP 

Designated  as  an  MDAP  by  the  MDA  or  its  dollar  value  is  estimated  by  OUSD 
(AT&L)  to  require  an  eventual  total  expenditure  for  RDT&E  of  more  than  $365 
million  in  fiscal  year  (FY)  2000  constant  dollars  or,  for  procurement,  of  more  than 
$2,190  billion  in  FY2000  constant  dollars. 

All  MDAPs  are  in  ACAT  1. 

MDAP  Decision  Authority. 

ACAT  ID:OUSD(AT&L). 

ACAT  1C:  head  of  the  DoD  component  or,  if  delegated,  the  Component 

Acquisition  Executive  (CAE)  (not  further  delegable). 

MAIS 

Designated  by  the  MDA  as  a  MAIS  or  its  dollar  value  is  estimated  to  exceed 
$32  million  in  FY2000  constant  dollars  for  all  expenditures,  for  all  increments 
incurred  in  any  single  fiscal  year,  or  $126  million  in  FY2000  constant  dollars  for 
all  expenditures,  for  all  increments  incurred  from  the  beginning  of  the  materiel 
solution  analysis  phase  through  deployment  at  all  sites,  or$378  million  in  FY2000 
constant  dollars  for  all  expenditures,  for  all  increments  incurred  from  the 
beginning  of  the  materiel  solution  analysis  phase  through  sustainment  for  the 
estimated  useful  life  of  the  system. 

All  MAIS  are  in  ACAT  IA.MAIS  Decision  Authority. 

ACAT  1AM:  OUSD  (AT&L)  or  designee. 

ACAT  IAC:  head  of  the  DoD  component  or,  if  delegated,  the  CAE  (not  further 
delegable). 

Special  interest 
programs 

OUSD  (AT&L)  shall  designate  programs  as  special  interest  based  on  one  or  more 
of  the  following  factors: 
technological  complexity 
congressional  interest 
large  commitment  of  resources 

program  is  critical  to  achievement  of  a  capability  or  set  of  capabilities 
program  is  a  joint  program. 

MDAPs  and  MAIS  cannot  be  considered  special  interest  programs. 

Special  interest  programs  can  be  categorized  as  any  ACAT. 

ACAT  ID  or  IAM:MDA  is  OUSD  (AT&L)  or  designee. 

ACAT  1C  or  IAC:MDA  is  head  of  DoD  component  or,  if  designated,  CAE  ACAT  II  or 

III. 

MDA  is  head  of  DoD  component  or,  if  designated,  CAE. 

Pre-MDAP,  MAIS 

A  Pre-major  defense  acquisition  program  (or  pre-MAIS)  is  one  that  is  in  the 
materiel  solution  analysis  or  technology  development  phases  preceding  MS  B  of 
the  defense  acquisition  system  and  has  been  identified  to  have  the  potential  to 
become  an  MDAP. 

AC  AT  II 

Does  not  meet  criteria  for  MDAP. 

Major  system. 

Dollar  value:  estimated  by  the  DoD  component  head  to  require  an  eventual  total 
expenditure  for  RDT&E  of  more  than  $140  million  in  FY2000  constant  dollars, 
for  procurement  of  more  than  $660  million  in  FY2000  constant  dollars,  or  MDA 
designation  as  special  interest. 

Decision  authority  CAE  or  individual  designated  by  CAE  Automated  Information 
Systems  (AIS)  cannot  be  categorized  as  ACAT  II. 

AC  AT  III 

Does  not  meet  criteria  for  ACAT  II. 

AIS  that  is  not  MAIS. 

Decision  authority  designated  by  CAE. 
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Although  most  of  the  WSARA  legislation  and  the  codified  reporting  and  con¬ 
trol  requirements  apply  only  to  the  MDAPs,  WSARA  did  not  entirely  ignore  the  per¬ 
formance  of  smaller  programs.  Instead,  it  gave  the  newly  created  PARCA  office  the 
authority  to  issue  additional  guidance  on  reporting  and  control  requirements  for  other 
types  of  programs,  including  the  smallest  ACAT  II  and  III  programs,  as  necessary 
and  appropriate.  Specifically,  it  states  that  PARCA  is  responsible  for  “Issuing  poli¬ 
cies,  procedures,  and  guidance  governing  the  conduct  of  performance  assessments  and 
root  cause  analyses  by  the  military  departments  and  the  Defense  Agencies.”  This  gives 
PARCA  the  option  to  extend  more  rigorous,  MDAP-like  guidance  to  smaller  acquisi¬ 
tion  programs,  but  it  also  allows  PARCA  the  option  of  keeping  the  status  quo.  This 
responsibility  suggests  two  related  questions  that  PARCA  leaders  must  answer  and  that 
motivate  this  report:  First,  is  additional  guidance  needed?  Second,  if  so,  what  should 
this  guidance  look  like? 

Determining  the  appropriate  guidance  and  the  right  level  of  oversight  to  apply 
to  ACAT  IIs  is  important  because,  although  the  MDAPs  are  the  largest  programs  in 
terms  of  size,  non-MDAP  programs  make  up  a  much  larger  share  of  the  total  active 
acquisition  programs.  Table  3.2  shows  that  about  92  percent  of  all  programs  are  non- 
MDAPs  not  subject  to  centralized  OSD  oversight  or  existing  Nunn-McCurdy  report¬ 
ing  and  control  requirements.  Instead,  these  non-MDAP  programs  are  currently  man¬ 
aged  by  the  military  departments  and  the  defense  agencies  themselves,  each  of  which 


Table  3.2 

Program  Type:  By  the  Numbers 


Program  Type 

IAC 

ACAT 

Total 

1AM 

1C 

ID 

II 

III 

N/A 

(Blank) 

MAIS 

11 

14 

25 

Pre-MAIS 

2 

17 

19 

MDAP 

34 

56 

90 

Pre-MDAP 

4 

34 

38 

Special  interest 

1 

1 

50 

3 

55 

Other 

2 

2 

(Blank) 

110 

850 

1 

2 

963 

Total 

13 

32 

34 

61 

162 

850 

1 

39 

1,192 

SOURCES:  Derived  from  DAMIR,  "Active  Programs,"  November  28,  2011;  Assistant  Secretary  of  the 
Navy  (Research,  Development  &  Acquisition)  (ASN  [RDA])  "Active  Program  List,"  August  10,  2011;  and 
Assistant  Secretary  of  the  Army  for  Acquisition,  Logistics,  and  Technology  (ASA  [ALT]),  "Active  ACAT  II 
and  ACAT  III  Program  List,"  January  4,  2012. 

NOTE:  Blank  cells  indicate  no  program  in  category. 
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has  its  own  procedures  for  monitoring  program  health  and  dealing  with  cost  overruns 
and  schedule  slips.  As  shown  in  Figure  3.1,  the  military  departments  and  defense  agen¬ 
cies  currently  oversee  and  hold  milestone  decision  authorities  for  about  97  percent  of 
non-MDAPs,  or  89  percent  of  active  programs. 


Military  Department  Oversight  of  ACAT  II  Programs 

The  military  departments  have  formal  and  defined  systems  in  place  to  oversee  and 
monitor  their  non-MDAP  programs.  These  systems  are  similar  to  those  used  to  moni¬ 
tor  MDAPs  and  may  include  the  use  of  RCA  and  performance  assessment  (PA).  How¬ 
ever,  whereas  all  MDAP  programs  are  subject  to  these  guidelines,  only  certain  non- 
MDAPs  are  formally  reviewed.  In  each  case,  only  “selected  non-MDAP  programs” 
are  subject  to  formal,  rigorous  review  processes.  Each  military  department’s  specific 
methods  for  selecting  non-MDAPs  to  follow  the  more  stringent  oversight  and  report¬ 
ing  and  control  requirements  of  MDAPs  do  not  appear  to  be  formally  written  down 
in  a  central  location.  As  a  result,  it  is  still  not  clear  how  often  reviews  are  applied  to 
ACAT  IIs,  how  effective  they  are  in  identifying  or  stopping  cost  overruns  and  sched- 

Figure  3.1 

Program  Management  Responsibility 


I  I  DoD  agencies 
I  I  Navy 
H  Army 
I  I  Air  Force 


□  DoD  CIO 
ITT  OUSD  (AT&L) 


SOURCES:  Derived  from  DAMIR,  2011;  ASN  (RDA),  2011;  ASA  (ALT),  2012;  Department  of  the  Navy, 
"Implementation  and  Operation  of  the  Defense  Acquisition  System  and  the  Joint  Capabilities  and 
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ule  slippage,  and  how  frequently  ACAT  II  programs  experience  severe  cost  overruns 
and  schedule  slippage  that  might  be  prevented  by  more  effective  centralized  or  rigor¬ 
ous  oversight.  These  questions  must  be  answered  before  we  can  say  for  certain  whether 
additional  reporting  and  control  requirements  might  improve  the  overall  performance 
of  ACAT  II  programs. 

However,  each  military  department  has  a  set  of  documented  procedures  to  assess 
and  monitor  acquisition  programs — one  that  is  applied  to  MDAPs  and  at  least  some 
ACAT  IIs.  Each  military  department  also  handles  oversight  of  acquisition  programs 
slightly  differently.  The  following  sections  will  describe  some  of  the  frameworks  in 
place  that  are  designed  to  assess  program  health,  inform  senior  leaders,  and  correct 
deviations  in  programs  that  experience  cost  growth  and  schedule  slippage. 

Army 

The  Army  has  a  number  of  unique  structures  and  methods  for  monitoring  program 
health.  The  Army  Systems  Acquisition  Review  Council  (ASARC)  is  central  to  the 
Army’s  oversight  of  its  acquisition  programs.7 

ASARC  provides  senior-level  acquisition  managers  and  functional  principals  a 
forum  to  formally  review  a  program’s  progress  and  health  and  determine  if  it  is  ready  to 
move  on  to  the  next  acquisition  phase.  ASARC  is  the  Army’s  review  board  for  MDAPs 
and  selected  ACAT  II  programs  where  the  Army  Acquisition  Executive  (AAE)  is  the 
MDA.  Case  by  case,  the  AAE  may  delegate  MDA  responsibility  to  the  Program  Exec¬ 
utive  Office  (PEO)  level,  in  which  case  an  ASARC  is  unnecessary  for  that  ACAT  II 
program.  ASARC  membership  consists  of  the  Assistant  Secretary  of  the  Army  (Acqui¬ 
sition,  Logistics,  and  Technology)  (ASA  [ALT]),  who  chairs  the  council,  Vice  Chief  of 
Staff  of  the  Army  (VCSA),  Deputy  Under  Secretary  of  the  Army  (Test  and  Evaluation) 
(DUSA  [T&E]),  Assistant  Secretary  of  the  Army  (Financial  Management  and  Comp¬ 
troller)  (ASA  [FMC]),  Assistant  Secretary  of  the  Army  (Installations  and  Environ¬ 
ment)  (ASA  [I&E]),  Assistant  Secretary  of  the  Army  (Manpower  and  Reserve  Affairs) 
(ASA  [M&RA]),  as  well  as  the  Deputy  Chief  of  Staff  (DCS)  from  all  Army  staff.  The 
ASARC  is  supported  by  Integrated  Product  Teams  (IPTs),  which  engage  in  the  day- 
to-day  oversight.  The  overarching  IPT  (OIPT),  which  is  chaired  by  the  ASA  (ALT) 
Deputy  Assistant  Secretary  of  the  Army  for  Acquisition  and  Systems  Management 
(ASM)  DASA  (ASM),  conducts  program  reviews  before  ASARC  and  works  to  resolve 
programmatic  issues  that  cannot  be  resolved  at  lower  levels. 

ASARC  gets  data  on  program  health  from  documentation  produced  by  program 
managers.  Some  of  this  documentation  is  statutory  and  cannot  be  tailored  for  a  specific 
program.  Other  pieces  of  documentation  allow  for  alterations  that  facilitate  the  inclu¬ 
sion  of  program-specific  information.  The  PM  can  engage  the  OIPT  to  determine  the 


7  Department  of  the  Army,  Army  Acquisition  Procedures,  Headquarters,  Pamphlet  70-3,  Washington,  D.C., 
January  28,  2008. 
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optimal  way  to  report  program  information  via  documentation.  Oversight  documents 
include  the  Acquisition  Program  Baseline  (APB),  Acquisition  Decision  Memorandum 
(ADM),  Army  Cost  Position  (ACP),  Acquisition  Strategy  Report  (ASR),  and  dozens 
of  others.  These  are  supplemented  by  supporting  documents,  documents  specific  to 
congressional  and  Defense  Acquisition  Board  (DAB)  oversight  reporting,  as  well  as 
program-specific  documents. 

ASARC  conducts  two  types  of  reviews:  program  reviews  (PRs)  and  milestone 
decision  reviews  (MDRs).  ASARC  IPTs  support  the  program  office  in  preparing  for 
these  reviews,  including  assisting  with  the  production  of  documentation.  MDRs  are 
typically  more  extensive  than  PRs.  The  outcomes  of  a  MDR  are  typically  an  ADM  or 
approved  APB. 

Once  an  APB  has  been  established,  cost,  schedule,  and  performance  parameters 
are  set.  A  breach  occurs  when  the  program  breaks  the  thresholds  set  by  these  param¬ 
eters.  There  are  two  types  of  breaches.  A  programmatic  breach  occurs  when  factors 
outside  the  control  of  the  program  office  cause  a  threshold  to  be  broken.  These  include 
guidance  from  above  the  program  office  level,  requirement  revisions,  program  restruc¬ 
turing,  and  doctrinal  changes.  A  fact-of-life  breach  is  due  to  internal  factors,  such  as 
technical  or  managerial  problems  leading  to  a  breach  of  the  program’s  cost,  schedule, 
or  performance. 

A  breach  sets  several  processes  in  motion.  A  PM  review  is  automatically  trig¬ 
gered.  The  PM  must  notify  the  MDA  in  a  timely  manner  of  the  breach.  The  notifica¬ 
tion  should  be  concise  and  contain  courses  of  action  necessary  to  correct  and  reset  the 
thresholds  that  were  breached. 

Air  Force 

Air  Force  oversight  is  part  of  a  larger  acquisition  and  sustainment  concept  known  as 
Integrated  Life  Cycle  Management  (ILCM).  ILCM  is  a  collection  of  processes  designed 
to  seamlessly  integrate  all  portions  of  a  life  cycle  from  inception,  development,  acquisi¬ 
tion,  fielding,  and  sustainment.8 

Within  the  ILCM  construct,  the  Air  Force  uses  Air  Force  Review  Boards  (AFRBs) 
and  Acquisition  Strategy  Panels  (ASPs)  to  monitor  program  health  and  assist  the 
MDA  in  milestone  decisions.  AFRBs  are  forums  chaired  by  the  service  acquisition 
executive  (SAE)  for  conducting  reviews  before  milestone  decisions.  They  are  required 
for  nondelegated  ACAT  II  programs.  ASPs  are  forums  to  assist  the  SAE  and  other 
MDAs.  They  ensure  that  the  best  recommendation  is  made  to  these  bodies. 

The  PM  is  responsible  for  supplying  all  statutory  documentation.  These  docu¬ 
ments  include  the  Life  Cycle  Management  Plan  (LCMP),  Information  Support  Plan 
(ISP),  Systems  Engineering  Plan  (SEP),  Test  and  Evaluation  Master  Plan  (TEMP),  and 
Risk  Management  Plan  (RMP),  among  others.  The  LCMP  contains  the  acquisition 


Air  Force  Instruction  63-101,  Acquisition  and  Sustainment  Life  Cycle  Management,  April  8,  2009. 
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and  sustainment  strategy  for  the  entire  life  cycle  of  the  system.  Although  these  docu¬ 
ments  are  required  by  law,  the  format  and  level  of  detail  are  unspecified,  allowing  the 
PM  latitude  in  the  creation  of  documentation. 

The  Air  Force  does  not  have  specific  requirements  to  handle  breaches  in  place 
beyond  those  laid  out  in  DoDD  5000.  They  do  house  responsibility  for  APB  breach 
reporting  with  the  Deputy  Assistant  Secretary  for  Acquisition  Integration  and  require 
that  PMs  immediately  notify  the  PEO  of  any  breach. 

Navy 

Just  as  the  Army  and  Air  Force  have  unique  oversight  processes  to  determine  program 
health  and  make  milestone  decisions,  the  Navy  uses  its  own  two-pass  and  six-gate 
review  processes  for  acquisition  programs.  The  goal  of  these  processes  is  to  ensure  that 
acquisition  aligns  with  Navy  requirements.9  However,  not  all  programs  are  subject  to 
the  review.  A  brief  overview  of  this  process  follows. 

The  process  is  divided  into  two  passes,  with  three  gates  per  pass.  The  first  pass 
encompasses  the  materiel  development  decision  and  materiel  solution  analysis  phase. 
It  is  led  by  the  Chief  Naval  Officer  (CNO)  or  the  Commandant  of  the  Marine  Corps 
(CMC),  and  the  three  gates  during  this  pass  all  deal  with  requirements.  The  first  gate 
review  grants  authority  for  the  submission  of  an  Initial  Capabilities  Document  (ICD) 
for  joint  review.  This  gate  also  confirms  the  plan  for  an  Analysis  of  Alternatives  (AoA). 
The  second  gate  occurs  after  the  AoA  completion  but  before  documentation  necessary 
at  MS  A  is  submitted  to  the  MDA.  The  AoA  is  reviewed  at  this  gate,  and  the  initial  key 
performance  parameters  (KPPs)  and  key  system  attributes  (KSAs)  are  confirmed  for 
use  in  the  Capability  Development  Document  (CDD).  The  third  gate  review  permits 
the  CDD  to  be  service- approved  and  provides  full  funding  certification  for  MS  A. 

The  second  pass  covers  the  MS  A  and  technology  development  phase  and  the 
MS  B  and  EMD  phase.  This  pass,  which  also  includes  three  gate  reviews,  is  led  by  the 
CAE.  Instead  of  requirements  gates  as  in  pass  one,  these  gate  reviews  are  acquisition- 
focused.  The  fourth  of  the  six  gates  approves  the  formal  system  design  specification 
(SDS)  and  allows  the  program  to  move  to  MS  B.  The  fifth  gate  review  ensures  that  the 
Navy  has  heeded  all  recommendations  from  the  MDA  and  provides  full  funding  cer¬ 
tification  for  MS  B.  This  gate  review  can  occur  before,  during,  or  after  the  MS  B  deci¬ 
sion,  depending  on  the  program’s  acquisition  strategy.  The  final  gate  review  assesses 
overall  program  health  after  the  award  of  the  EMD  contract. 

Should  a  Navy  program  breach  the  cost,  schedule,  or  performance  thresholds  set 
in  the  APB,  the  MDA  must  be  notified.  The  program  office  has  30  days  to  supply  a 
report  outlining  the  cause  of  the  breach  and  planned  action  to  rectify  the  situation. 


9  SECNAVINST  5000. 2E,  Department  of  the  Navy  Implementation  and  Operation  of  the  Defense  Acquisition 
Systems  and  the  Joint  Capabilities  Integration  and  Development  Systems,  September  1,  2011. 
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Summary 

The  mechanisms  established  by  the  military  departments  to  monitor  cost  growth  and 
other  problems  acquisition  face  are  fairly  extensive  and  include  centralized  and  decen¬ 
tralized  oversight  processes  as  well  as  reporting  and  control  requirements  designed 
to  identify  poor  performers  and  the  root  causes  of  their  problems.  However,  these 
requirements  apply  primarily  to  MDAPs,  leaving  smaller  ACAT  II  programs  with  less 
guidance.  The  military  departments  monitor  ACAT  II  programs  largely  on  their  own, 
and  they  have  systems  in  place  to  evaluate  and  track  program  performance.  However, 
although  non-MDAPs  make  up  the  majority  of  acquisition  programs  in  terms  of  total 
number,  only  some  of  these  programs  are  subject  to  formal  review.  Furthermore,  it  is 
not  entirely  clear  how  the  military  departments  choose  which  programs  are  subject  to 
these  reviews.  These  observations  combined  with  continued  attention  to  issues  of  cost 
growth  in  acquisition  programs  among  policymakers  raise  the  question  of  whether 
additional  guidance  and  requirements  should  be  applied  to  the  ACAT  II  programs. 


Assessing  ACAT  II  Programs 

With  this  background,  we  now  turn  to  our  program  assessment.  The  section  begins 
with  a  discussion  of  our  methodology,  followed  by  a  brief  narrative  and  program  assess¬ 
ment  of  the  ACAT  II  programs  that  we  selected  from  across  military  departments 
and  procurement  categories.  It  concludes  with  an  overall  assessment  of  ACAT  II  pro¬ 
gram  performance,  including  a  discussion  of  common  problems  and  the  scope  of  these 
problems. 

Approach 

Assessing  the  performance  of  non-MDAPs  is  not  straightforward,  because  data  are 
not  easy  to  find  or  to  use.  There  is  no  comprehensive  list  of  non-MDAPs  in  common 
acquisition  databases,  such  as  DAMIR.  Instead,  we  rely  on  budget  Exhibit  P-ls  and 
P-40s  provided  by  the  military  departments  and  included  in  the  President’s  official 
budget  justifications,  as  well  as  on  congressional  hearings  and  testimony  to  construct 
program  narratives. 

Exhibit  P-ls  and  P-40s  include  information  on  program  size  and  budget  activity 
(procurement  program  category).  Exhibit  P-1  includes  top-level  summary  informa¬ 
tion  on  cost  and  quantity  for  the  past,  current,  and  requested  fiscal  year.  We  used 
Exhibit  P-1  to  select  ten  ACAT  II  programs  from  each  procurement  program  category 
across  all  three  military  departments.  In  each  case,  we  selected  from  among  the  largest 
ACAT  II  programs  that  had  been  active  within  the  last  five  years,  but  we  excluded  any 
programs  that  had  been  MDAPs. 

Exhibit  P-40  provides  more  detailed  data  on  cost  and  quantity,  including  pro¬ 
curement  of  non-end  items,  such  as  initial  spares  and  engineering  services.  Exhibit 
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P-40  also  provides  a  basic  program  history  that  summarizes  the  program’s  development 
and  sometimes  identifies  performance  or  program  health  issues,  as  well  as  explanations 
for  revisions  to  past  year  estimates  or  future  projections.  We  used  Exhibit  P-40s  pro¬ 
vided  by  the  military  departments  to  collect  available  data  on  cost  and  quantity  for  all 
programs.  Exhibit  P-40s  contain  at  least  three  and  sometimes  as  many  as  seven  years 
of  data.  Values  two  years  before  the  budget  request  are  actual  values;  values  from  the 
year  before  the  budget  request  are  being  executed  and  subject  to  change;  and  values 
from  the  year  of  the  budget  request  are  subject  to  change.  These  values  are  available  for 
every  program.  Estimates  for  future  year  requests/funding  often  appear  for  the  next 
four  years,  which  constitute  the  Future  Years  Defense  Program  (FYDP).  The  exhib¬ 
its  include  information  on  cost  and  quantity  and  details  on  what  the  money  for  each 
program  specifically  procures  (e.g.,  hardware,  peripherals,  service  contracts,  training) 
in  a  given  year.  In  addition,  Exhibit  P-40  includes  a  description  of  the  program  itself, 
its  history,  and  sometimes  description  of  problems  or  the  rationale  for  revisions  in  cost 
or  quantity.  By  collecting  and  merging  data  from  Exhibit  P-40  for  each  budget  year, 
we  can  construct  time  series  data  of  unit  cost  and  quantity  over  a  program’s  life  span. 
Figures  3.2  and  3.3  show  examples  of  what  Exhibit  P-1  and  Exhibit  P-40  tables  look 
like  when  published. 

To  supplement  the  information  included  in  Exhibit  P-40,  we  used  congressional 
hearings,  testimony,  and  markups  by  the  House  Armed  Services  Committee  (HASC), 
the  Senate  Armed  Services  Committee  (SASC),  the  House  Appropriations  Committee 

Figure  3.2 
Sample  Exhibit  P-1 
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Figure  3.3 

Sample  Exhibit  P-40 
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Exhibit  P-40,  Budget  Item  Justification  Sheet  iDate: 

|  February  2006 

Appropriation  /  Budget  Activity  /  Serial  No:  P-1  Item  Nomenclature 

ProcwnremorWATCV.  Army  1  Tracked  ccmb«  vetodts  STP.YKE?  VEHICLE  (G8S100) 

Program  Elements  for  Code  B  Items:  Code  Other  Related  Program  Elements 

0603653AC03  |  B 

Prior  Years 

FY  2005 

FY  2006 

FY  2007 

FY  2008 

FY  2009 

FY  2010 

FY  2011 

To  Complete 

Total  Prog 

ProcQty 

1446 

575 

340 

100 

129 

10 

10 

10 

2620 

Gross  Cost 

32687 

1434.0 

1049  4 

7960 

8041 

289  1 

3906 

327.0 

938.7 

9297.6 

Less  PY  Adv  Proc 

Phis  CY  Adv  Proc 

Net  Proc  PI 

32687 

1434  0 

1049  4 

7960 

8041 

289  1 

3906 

327.0 

938.7 

9297.6 

Initial  Spares 

Total  Proc  Cost 

32687 

1434.0 

1049.4 

796.0 

8041 

289.1 

390.6 

327.0 

938.7 

9297.6 

Flyaway  U/C 

Weapon  System  Proc  U/C 

23 

2.5 

31 

80 

62 

289 

391 

32.7 

3.5 

Description: 

A  dynamic  asymmetric  threat  and  operational  environment  demands  full  spectrum,  strategically  responsive,  agile  and  dominant  land  forces.  Immediate  response  by  a  lethal,  versatile,  tactically  agile 
joint  force  capable  of  operational  maneuver  is  essential  to  fulfilling  the  Army’s  Warfightmg  needs  The  Stryker-equipped  BCT  is  this  force  The  use  of  a  common  platfotm'common  chassis  design 
reduces  requirements  for  repair  parts  and  logistics  support  m  the  area  of  operations 

The  Bngade  Combat  Team  (BCT).  equipped  with  Stryker  Vehicles  can  be  used  across  the  full  spectrum  of  military  operations  The  Stryker  Brigade  Combat  Team  deploys  rapidly  and  conducts 
effective  combat  operations  immediately  on  arrival  to  prevent  contain,  stabilize,  or  resolve  a  conflict  through  shaping  and  decisive  operations  Configurations  of  Stryker  Vehicles  are: 

Infantry  Carrier:  The  Infantry  Carrier  Vehicle  (ICV)  carries  an  infantry  squad  with  m  dividual  equipment  The  ICV  also  provides  protected  transport  and  supporting  fires  for  the  in  fanny  squad 
during  dismounted  assault. 

Reconnaissance  Vehicle  The  Reconnaissance  Vehicle  (RV)  provides  force  situational  awareness,  gathenng  and  transmitting  real  tune  intelligence  while  moving  throughout  the  battlefield  m  close, 
complex,  and  urban  terrain. 

Mobile  Gun  System:  The  Mobile  Gun  System  (MGS)  supports  dismounted  infantry  and  engages  the  enemy  m  close  combat  m  order  to  clear  opposition  and  permit  rapid  movement  allowing  the 
force  to  maintain  die  initiative,  occupy  and'or  secure  key  objectives,  and  defeat  strong  points. 

Mortar  Carrier:  The  Mortar  Carrier  (MC)  will  siqiport  infantry  units  with  screening  obscurants,  suppressive  forces  and  on-call  supporting  fires  1 30mm, 8 1  mm.  and  60mm  mortars  provide 
complimentary  capabilities  with  responsive,  accurate  and  lethal  indirect  fire  support  to  the  dismounted  infantry  assault 

nandes’s  Vehicle:  The  Commander’s  Vehicle  (CV)  provides  the  bngade  commanders  and  infantry  battalion  commanders  with  the  means  to  receive  information  and  data,  analyze,  prepare  and 
nit  data,  and  control  the  forces/functions  carrying  out  combat  missions 
Fire  Support  Vehicle:  The  Fire  Support  Vehicle  (FSV)  provides  automated  enhanced  surveillance,  target  acquisition,  target  identification,  target  tracking,  target  designation  position  location  and 
nmications  functionality  Targets  will  be  transmitted  instantly  to  die  fire  support  system  and  shooter 
Engineer  Squad  Vehicle:  The  Engineer  Squad  Vehicle  (ESV)  provides  maneuver  'mobility  support  capabilities  which  include  obstacle  dealing,  in-stnde  breaching  of  surface  mines,  proofing  of 
subsurface  mines,  and  smoke  eeneration  for  local  protection 
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(HAC),  and  the  Senate  Appropriations  Committee  (SAC).  These  documents  provide 
insight  into  congressional  support  for  each  program  and,  sometimes,  congressional 
concerns  about  cost,  schedule,  or  performance.  Instances  where  the  HASC  or  SASC  or 
another  congressional  committee  recommends  more  or  less  than  the  requested  amount 
of  funding  are  also  valuable  and  provide  insight  into  the  health  of  the  program. 

Data  on  RDT&E  costs  are  located  in  Exhibit  R-ls.  For  ACAT  II  and  smaller 
programs,  these  reports  often  provide  data  aggregated  to  a  level  that  makes  it  difficult 
to  attach  specific  research  costs  to  specific  programs. 

Graphs  and  Program  Assessments 

Using  the  time  series  data  on  program  cost  and  quantity,  we  graphed  annualized  unit 
cost  against  annualized  quantity.  We  considered  using  graphs  to  compare  average  unit 
cost  and  cumulative  quantity  procured,  but  we  found  these  metrics  less  useful  for  the 
purposes  of  this  research  effort,  because  they  smooth  over  and  average  out  the  variation 
in  unit  cost  and  quantity  procured  over  time  that  we  are  most  interested  in  assessing  as 
an  indicator  of  program  health.  In  contrast,  graphs  using  annualized  quantity  and  unit 
cost  provide  more  valuable  insight  into  changes  in  program  health  and  performance 
over  time  and  also  capture  the  overall  stability  of  the  program  over  time.  We  use  the 
same  approach  in  our  assessment  of  selected  set  of  MDAPs,  making  our  evaluations  of 
the  two  sets  of  programs  generally  comparable.  Although  the  graphs  use  actual  values, 
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we  also  collected  projected  cost  and  quantity  for  each  available  year  and  program,  so 
that  we  could  assess  the  frequency  and  size  of  revisions. 

We  assessed  these  graphs  looking  at  two  primary  criteria.  First,  we  looked  for 
large  revisions  and  sharp  fluctuations  in  quantity  and  cost,  since  these  indicate  weak 
program  performance.  Second,  we  looked  for  slow  growth  over  time.  The  rationale  for 
caring  about  this  second  type  of  growth  is  the  same  as  that  behind  the  Nunn-McCurdy 
breach.  Even  programs  with  small  year-to-year  increases  in  unit  cost  may  show  signifi¬ 
cant  and  problematic  cost  growth  over  a  full  program  life  span.  To  explain  discrepan¬ 
cies  and  potential  problems,  we  matched  the  narrative  information  on  each  program  to 
the  unit  cost  graphs.  In  some  cases  Congress  or  the  relevant  military  department  pro¬ 
vided  an  explanation  for  cost  growth,  but  in  many  cases,  no  explanation  was  provided. 

In  addition  to  looking  at  cost  growth,  we  also  considered  whether  there  was  a 
clear  relationship  between  unit  cost  and  quantity  and  whether  specific  contract,  devel¬ 
opment,  or  other  program  characteristics  appeared  to  explain  the  overall  success  or 
failure  of  the  program. 

Challenges 

We  faced  some  challenges  in  the  assessment  of  ACAT  II  programs.  First,  data  on  ACAT 
II  cost  and  quantity  are  provided  in  budget  requests  that  are  published  as  annual  vol¬ 
umes,  which  means  that  the  time  series  must  be  constructed  manually.  Inconsisten¬ 
cies  and  gaps  are  common,  and  programs  sometimes  change  names  as  they  pass  key 
milestones,  which  makes  tracking  their  development  over  time  somewhat  challenging. 
Some  programs  are  also  funded  jointly  by  several  components,  which  creates  addi¬ 
tional  challenges  and  the  potential  for  discrepancies.  Finally,  program  funds  may  pur¬ 
chase  not  only  the  hardware  but  also  peripherals  and  service  contracts.  These  costs  are 
relevant  to  the  total  program  cost  but  must  be  factored  out  to  generate  comparable 
unit  cost  estimates  over  time.  We  were  able  to  conduct  these  cost  adjustments  where 
necessary,  using  supplemental  information  in  Exhibit  P-40s.  However,  one  result  of 
these  many  challenges  is  that  our  calculated  unit  costs  should  be  considered  estimates 
rather  than  precise  values.  We  are  confident  these  estimates  are  sufficient  to  support 
our  assessment  of  program  health. 


Assessing  ACAT  II  Programs:  Program  Narratives  and  Results 

Overview  and  Summary 

Table  3.3  lists  the  ACAT  II  programs  included  in  our  selected  set  of  along  with  the 
service  and  the  procurement  type.  Although  we  considered  ten  programs,  we  have  12 
program  results  because  we  consider  the  Hellfire  missile  program  separately  for  the 
Army,  Navy,  and  Air  Force.  We  selected  our  ten  programs  carefully  from  among  the 
set  of  available  ACAT  II  programs,  specifically  choosing  those  with  the  largest  funding 
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Table  3.3 

ACAT  II  Programs  Selected 


Program 

Service 

Procurement  Type 

C-37A 

Air  Force 

Aircraft 

Hellfire  missile 

Air  Force,  Army,  Navy 

Air  Force  and  Army:  missiles 

Navy:  weapons 

MQ-8 

Navy 

Aircraft 

RQ-11  Raven 

Army 

Aircraft 

Assault  Breacher  Vehicle 

Army,  Marine  Corps 

Weapons  and  tracked  combat  vehicles 

MTVR 

Marine  Corps 

Weapons  and  tracked  combat  vehicles 

RAM 

Navy 

Weapons 

LCAC  SLEP 

Navy 

Shipbuilding  and  conversion 

M2  .50  caliber  roll 

Army 

Weapons  and  tracked  combat  vehicles 

MLRS  RRPR 

Army 

Missiles 

allocations  in  official  budget  documents  over  the  past  five  years.  This  assured  that  they 
would  be  more  like  MDAPs  than  like  smaller  programs.  Second,  we  aimed  to  choose 
at  least  one  program  from  each  service  and  procurement  category  to  build  a  representa¬ 
tive  set  of  systems. 

Despite  our  efforts,  our  selection  will  still  present  some  limitations.  Because  we 
assess  only  a  select  set  of  programs,  our  assessments  and  observations  apply  primarily  to 
the  specific  programs  that  we  have  included.  However,  our  sampling  approach  should 
produce  a  selected  set  that  is  representative,  allowing  us  to  generalize  carefully  to  other 
ACAT  II  programs.  In  addition,  because  we  select  the  largest  ACAT  II  programs  from 
across  procurement  types,  our  selected  set  of  programs  should  also  be  reasonably  com¬ 
parable  to  most  MDAPs  on  such  dimensions  as  technological  complexity.  This  is  a 
point  we  refer  to  again  in  our  assessment  of  MDAPs. 

To  preview  the  results  of  our  assessment,  we  found  that,  overall,  the  ACAT  II  pro¬ 
grams  perform  reasonably  well,  without  significant  evidence  of  runaway  cost  growth  or 
schedule  slippage.  Table  3.4  summarizes  program  performance  and  gives  a  short  expla¬ 
nation  of  concerns  where  they  exist.  Of  the  programs  we  consider,  we  find  that  four 
show  no  reason  for  concern,  five  have  some  issues  of  minor  concern,  and  three  warrant 
some  more  significant  concerns.  Across  the  board,  instability  in  unit  cost  and  quantity 
appears  more  problematic  than  cost  growth  or  massive  schedule  delays.  Instability  is 
particularly  likely  early  in  the  lives  of  new  ACAT  II  programs,  but  this  instability  is  not 
crippling,  and  programs  are  often  able  to  stabilize  over  time.  There  also  seems  to  be  a 
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Table  3.4 

ACAT  II  Performance  Assessments 


No  Serious 
Concerns 

Minor  Concern 

Some  Concerns 

MLRS  RRPR 

C37A:  Small  quantity  purchased  contributes 
to  large  fluctuations  in  unit  cost;  details  on 
leasing  program  limited 

MTVR:  steadily  growing  unit  costs, 
perhaps  as  a  result  of  multiple 
variants 

LCAC  SLEP 

ABV:  cross-service  discrepancies  and 
projected  cost  growth  out  to  FY2016 

RAM:  some  instability  and 
significant  projected  cost  growth 

M2  .50  caliber 

Army  He  1  If i re:  fluctuation  in  unit  costs,  costs 
higher  than  in  the  Navy 

Navy  Hellfire 

Air  Force  Hellfire:  Instability  in  unit  cost 
despite  off-the-shelf  purchase 

MQ8:  initial  unit  cost  growth  and  schedule 
delays  resulting  from  development 
problems 

RQ11:lnstabilty  in  unit  cost  and  quantity 
procured,  projected  cost  growth  in  the  long 
run 

NOTE:  We  characterize  the  Navy  He  I  If  i  re  program  as  one  with  no  serious  problems;  however,  we  discuss 
it  with  the  Army  and  Air  Force  programs  for  purposes  of  comparison. 


clear  relationship  between  unit  cost  and  quantity.  Much  of  the  cost  growth  that  we  do 
observe  occurs  because  of  downward  revisions  in  procurement  quantities — revisions 
that  likely  reflect  changes  in  demand  associated  with  the  contingencies  in  Iraq  and 
Afghanistan.  It  is  important  to  note  that  although  we  consider  fluctuation  in  quantity 
as  a  measure  of  program  health,  it  may  also  reflect  factors  outside  the  control  of  pro¬ 
gram  managers.  Finally,  although  there  are  instances  where  delays  and  development 
issues  affect  ACAT  II  program  performance  early  in  a  program’s  life  span  and  during 
modernization,  there  is  little  evidence  of  major  schedule  slippage  once  the  program 
enters  production.  Observed  delays  are  often  linked  to  modernizations  and  technologi¬ 
cal  advances.  However,  there  are  also  many  instances  of  rapidly  accelerated  time  lines. 
In  other  cases,  cost  growth  exists  only  in  the  projected  out-years.  This  means  that  it  is 
not  certain  that  this  cost  growth  will  materialize.  Finally,  differences  across  military 
departments  in  terms  of  program  performance  may  warrant  additional  investigation. 

Program  Narratives  and  Assessments 

In  this  section,  we  present  the  results  of  our  ACAT  II  program  assessment.  We  include 
a  brief  history  of  the  program,  its  major  milestones,  development  process,  and  any  sig¬ 
nificant  problems,  as  well  as  a  graph  showing  unit  cost  and  annualized  quantity  over 
the  life  of  the  program. 
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No  Serious  Concerns 

Multiple  Launch  Rocket  System,  Army,  Weapons  Procurement 

The  Multiple  Launch  Rocket  System  (MLRS)  Reduced-Range  Practice  Rocket  (RRPR) 
is  a  training  rocket  allocated  to  Army  Active  Duty  and  Reserve  MLRS  units.  It  is  the 
only  live  fire  training  rocket  or  missile  used  by  all  U.S.  Army  Field  Artillery  rocket  and 
missile  units/crews  and  ensures  that  these  units  achieve  and  maintain  combat  readi¬ 
ness  in  the  Global  War  on  Terror  (GWOT).  The  RRPR  supports  Army  modularity, 
since  the  High  Mobility  Artillery  Rocket  System  (HIMARS)  and  M270A1  battalions, 
which  use  the  rocket  for  training  purposes,  are  organic  and  attached  to  modular  fires 
brigades  supporting  Brigade  Combat  Teams  (BCTs),  Joint  Expeditionary  Force,  and 
Joint  Special  Operations  Force  (JSOF)  combatant  commands.10 

The  MLRS  RRPR  has  been  in  inventory  since  1993  with  the  last  U.S.  procure¬ 
ment  in  FY1995,  when  the  Army  decided  to  terminate  funding  for  MLRS  production. 
This  decision  resulted  because  the  Army  had  sufficient  tactical  rockets  to  meet  any 
anticipated  military  contingency  and  enough  RRPRs  to  meet  the  training  require¬ 
ments  for  Active  and  Reserve  component  forces  for  the  next  ten  years.  However,  the 
Senate  Appropriations  Committee  expressed  some  concern  about  the  need  to  maintain 
the  MLRS  production  base  until  the  extended-range  (ER)  rocket  was  to  begin  produc¬ 
tion  in  FY1997.* 11  The  House  expressed  a  similar  concern  and  recommended  that  the 
Army  continue  to  procure  the  RRPR  at  a  rate  of  no  more  than  500  per  month  while 
production  transitioned  to  the  ER  version  of  the  rocket,  and  production  level  was  sus¬ 
tained  to  cost-effectively  retain  key  elements  of  the  industrial  base.12  No  additional 
procurement  was  made  after  FY1995. 

By  FY2003,  the  existing  stock  of  these  training  rockets  had  been  reduced  as  a 
result  of  training  consumption,  and  additional  procurement  was  needed  to  preclude 
stockpile  depletion  and  sustain  adequate  stockpile  margins.  Reprocurement  was  espe¬ 
cially  important,  as  the  wars  in  Iraq  and  Afghanistan  intensified  and  live  fire  train¬ 
ing  became  increasingly  important.  Reprocurement  to  maintain  the  practice  rocket 
inventory  for  Standards  in  Training  Commission  (STRC)  requirements  began  in 
FY2003.  The  annual  requirement  for  these  missiles  in  FY2006,  according  to  STRC, 
was  approximately  4,800  per  year.  Starting  in  FY  2010,  the  system  was  known  as  the 
MLRS  low-cost  reduced-range  practice  rocket  (LCRRPR). 

The  graph  comparing  unit  cost  and  quantity  for  the  MLRS  RRPR  (Figure  3.4) 
program  shows  no  real  cause  for  concern.  The  program  has  not  experienced  significant 
instability  or  long-term  unit  cost  growth.  There  has  been  a  relatively  consistent  rela- 


10  Department  of  the  Army,  FY2013  Budget  Estimates  Justification  Book,  Missile  Procurement,  February  2012b. 

11  U.S.  Senate,  Department  of  Defense  Appropriations  Bill  1995  Report,  Washington,  D.C.,  U.S.  Government 
Printing  Office,  1994. 

12  U.S.  House  of  Representatives,  Department  of  Defense  Appropriations  Bill  1995  Report,  Washington,  D.C., 
U.S.  Government  Printing  Office,  1994b. 
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Figure  3.4 

Unit  Cost  and  Quantity,  MRLS  RRPR 
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tionship  between  unit  cost  and  quantity,  but  even  large  changes  in  quantity  procured 
have  little  effect  on  unit  cost.  After  the  first  year  of  reprocurement,  quantity  procured 
fell  and  unit  cost  rose  slightly.  Unit  cost  was  lowest  in  2003  but  rose  with  significantly 
reduced  quantities  purchased  in  2004,  2005,  and  2006.  It  is  worth  noting  that  the 
FY2004  budget  request  projected  significantly  larger  purchases  in  both  FY2004  and 
FY2005  (of  2,934  and  3,054,  respectively).  These  numbers  were  revised  downward  in 
FY2005.  The  Army  did  not  provide  a  reason  for  the  lower  quantity  procured  in  these 
years.  The  program  has  been  largely  stable  since  FY2007  and  is  projected  to  remain 
stable  in  the  near  future. 

The  lack  of  significant  problems  associated  with  this  program  may  reflect  the  fact 
that  the  MLRS  RRPR  was  not  a  new  item  but  a  fully  mature  item  that  the  Army  had 
already  developed.  The  FY2003  procurement  was  a  “restart”  rather  than  a  new  pro¬ 
gram,  so  there  were  no  initial  development  costs,  production  line  restarts  were  clearly 
insignificant,  and  the  program  could  enter  quickly  into  full-rate  production.  Also 
important  is  that  projected  stability  in  the  future  depends  on  continued  demand  for 
the  training  rockets.  If  demand  falls  with  the  end  of  the  wars  in  Afghanistan  and  Iraq, 
then  unit  cost  may  rise. 

LCAC  SLEP,  Navy,  Shipbuilding  and  Conversion 

TCAC  is  a  Navy  air-cushion  vehicle  for  use  in  amphibious  operations.  Ninety  craft 
were  constructed  by  Textron  Marine  and  Land  Systems  and  Avondale  Gulfport 
and  delivered  from  1984  to  1987.  Seventy-three  vehicles  have  received  or  are  set  to 
receive  the  Service  Life  Extension  Program  (SLEP).  Conversions  should  be  completed 
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by  20 18. According  to  the  FY2013  Navy  Shipbuilding  and  Conversion  Justification 
Book,  the  SLEP  “extends  the  craft  service  life  from  twenty  years  to  thirty  years.  The 
new  hull  incorporates  four  modifications:  additional  internal  compartmentation  to 
increase  cargo  carrying  capacity;  a  modified  fuel  system  to  increase  range;  improved 
skirt  attachments  to  reduce  maintenance  and  deep  skirt  to  improve  performance  and 
maximize  safety.  The  SLEP  will  also  include  the  C4N  electronic  suite  replacement  as 
well  as  a  modified  set  of  TF40B  engines,  designated  ETF40B.”13 

Congress  consistently  supported  the  program,  even  admonishing  the  Navy  for 
being  slow  to  address  the  corrosion  issues  that  would  reduce  the  life  cycle  of  the  LCAC 
below  its  projected  20  years.  Congress  was  also  critical  of  the  Navy  for  how  it  con¬ 
sidered  the  best  way  to  maintain  the  LCAC  and  its  production  line  over  the  service 
life  of  the  vehicles.  Although  the  Navy  did  not  request  any  funding  for  the  program 
until  FY1999  (in  a  mandated  report  on  SLEP  submitted  in  October  1996), 14  Congress 
interjected  and  added  $2.9  million  in  FY1997  and  $19.5  million  in  FY1998.  After  an 
initial  delay,  the  Navy  added  funding  for  advance  procurement  in  FY1999  to  initiate 
the  LCAC  SLEP  in  FY2000.  Developing  the  program  and  receiving  additional  fund¬ 
ing  from  Congress  was  vital,  because  of  the  Navy  and  Marine  Corps  heavy  reliance  on 
the  LCAC.  In  fact,  the  LCAC  was  responsible  for  transporting  95  percent  of  weapons 
and  tracked  combat  vehicles  in  a  Marine  Air-Ground  Task  Force  during  an  amphibi¬ 
ous  landing.15 

The  SLEP  construction  contract  option  exercise  for  the  FY  2000  and  FY2001 
LCACs  occurred  in  May  2001.  This  allowed  for  the  construction  start  of  two  LCAC 
SLEPs  in  FY2001.  An  FY2002  contract  provided  for  SLEP  construction  to  two  addi¬ 
tional  FY2002  craft.  The  two  additional  units  covered  by  the  FY2002  contract  was  the 
minimum  number  of  construction  starts  needed  to  preclude  a  significant  line  break.16 
The  contract  was  awarded  in  December  2002  (and  approved  on  April  10,  2002). 
During  the  early  2000s,  the  Marine  Corps  indicated  that  the  LCAC  SLEP  would  be 
a  high  priority  should  additional  funds  become  available.  Both  the  Flouse  and  Senate 
responded,  demonstrating  the  high  priority  of  the  program  and  confidence  in  its  per¬ 
formance.  During  the  FY2003  National  Defense  Authorization  Act  (NDAA)  markup, 
HASC  recommended  the  addition  of  $11  million  to  LCAC  SLEP  funding.  This  was 
to  allow  the  production  of  an  additional  buoyancy  box,  which  fixes  corrosion  prob¬ 
lems  and  adds  other  improvements  to  the  hull.  Adding  one  box  to  production  would 


13  Department  of  the  Navy,  FY2013  Budget  Estimates  Justification  Book,  Shipbuilding  and  Conversion,  February 
2012b. 

14  U.S.  House  of  Representatives,  Military  Procurement  Ship  Acquisition  and  Offshore  Based  Warfare:  Hearing 
Before  the  Armed  Services  Committee,  Washington,  D.C.,  U.S.  Government  Printing  Office,  1999a. 

15  U.S.  House  of  Representatives,  1999a. 

16  Department  of  the  Navy,  FY2003  Budget  Estimates  Justification  Book,  Shipbuilding  and  Conversion,  February 
2002b. 
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prevent  the  production  line  shutdown  for  the  buoyancy  box,  which  had  a  minimum 
production  rate  of  four  per  year.  SASC  went  one  step  further  by  recommending  the 
addition  of  $22  million  to  fund  one  additional  complete  conversion  during  FY2003, 
which  would  include  the  construction  of  a  buoyancy  box.  Around  this  time,  the  first 
converted  LCAC  SLEP  vehicle  was  delivered  on  schedule  and  on  time.  One  year  later, 
during  the  FY2004  NDAA  markup,  HASC  recommended  adding  $21.0  million  to 
boost  LCAC  SLEP  funding  from  $73.1  million  to  $94.1  million. 

Starting  in  FY2004,  some  modifications  to  SLEP  improved  the  program’s  effi¬ 
ciency  and  resulted  in  cost  savings.  Specifically,  rather  than  replacing  the  buoyancy 
box,  the  existing  box  was  to  be  refurbished  with  some  modifications.  According  to  the 
budget  justification  “this  change  allowed  construction  to  be  accomplished  near  the 
operating  units,  saving  transportation  as  well  as  disassembly  and  buoyancy  box  con¬ 
struction  costs,  while  still  achieving  the  same  operational  capabilities  and  service  life 
extension.”17  The  FY2005  appropriation  request  covered  the  full  rehabilitation  of  six 
reduced  operational  status  (ROS)  craft:  one  craft  each  in  FY2005  through  FY2009, 
but  two  craft  in  FY2007.  This  also  improved  the  efficiency  of  the  program  as  “reha¬ 
bilitating  the  ROS  craft  for  use  in  SLEP  will  avoid  taking  active  mission  capable  craft 
out  of  the  inventories  at  the  operating  units.”  In  FY2006,  SLEP  appropriations  covered 
additional  rehabilitation  of  LCAC  craft  through  FY2011,  along  with  “full  mission 
trainer  upgrades”  in  each  year.  Appropriations  requests  and  budget  projections  suggest 
that  SLEP  will  remain  active  in  the  coming  fiscal  years.  Although  total  and  unit  cost 
may  be  revised,  no  significant  problems  or  line  breaks  are  expected. 

The  LCAC  SLEP  has  very  little  cause  for  concern.  Figure  3.5  shows  that,  despite 
some  fluctuation  in  quantity  procured  over  the  life  of  the  program,  the  unit  cost 
remains  fairly  stable  over  time  and  even  decreases  over  the  life  of  the  program.  This 
overall  decrease  suggests  learning  by  program  managers  and  the  discovery  and  exploi¬ 
tation  of  more  cost-  and  time-effective  processes.  In  fact,  there  are  many  examples  of 
adept  program  management,  including  conversions  completed  ahead  of  schedule  and 
cost  savings  from  revised  acquisition  and  procurement  costs,  saving  over  $100  million, 
which  led  to  a  DoD  Value  Engineering  Award.  Congressional  support  for  the  program 
is  another  sign  of  program  health. 

Once  again,  there  is  a  relationship  between  unit  cost  and  quantity,  with  unit  cost 
rising  when  quantity  falls  for  the  LCAC  SLEP.  Program  information  suggests  that 
four  conversions  are  the  minimum  required  to  sustain  existing  industrial  capacity.  The 
graph  here  appears  to  confirm  this,  and  unit  cost  rises  more  significantly  when  quan¬ 
tity  falls  below  four.  In  a  few  places,  cost  increases  may  warrant  additional  attention: 
A  12  percent  increase  from  FY2004  to  FY2005  and  a  projected  unit  cost  increase  from 
FY2012  to  FY2013  are  worth  noting  (although  this  has  yet  to  materialize). 


17  Department  of  the  Navy,  FY2005  Budget  Estimates  Justification  Book,  Shipbuilding  and  Conversion,  February 
2004. 
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Figure  3.5 

Unit  Cost  and  Quantity,  LCAC  SLEP 
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M2  .50  Caliber,  Army,  Ammunition  Procurement 

The  M2  .50  caliber  automatic  rifle  has  been  a  mainstay  of  the  U.S.  armed  forces  since 
the  1920s.  It  is 

an  automatic,  belt-fed,  recoil-operated,  and  air-cooled  crew-served  weapon.  It 
mounts  on  the  M3  and  XM205  tripods  and  on  most  vehicles,  and  serves  as  an 
anti-personnel  and  anti-aircraft  weapon.  It  is  highly  effective  against  light  armored 
vehicles,  low-  and  slow-flying  aircraft,  and  small  boats.  The  M2  provides  automatic 
weapon  suppressive  fire  for  offensive  and  defensive  purposes.  It  is  capable  of  single¬ 
shot  and  automatic  fire.18 

The  U.S.  military  is  but  one  of  the  many  buyers  of  the  M2,  which  is  manufac¬ 
tured  by  General  Dynamics  Armament  and  Technical  Products.  Other  recent  pur¬ 
chasers  include  Egypt,  the  Saudi  National  Guard,  Colombia,  Greece,  Djibouti,  Bah¬ 
rain,  Romania,  Jordan,  Chile,  and  15  U.S.  agencies,  including  the  Army. 

Army  procurement  of  the  M2  began  again  as  a  new  start  in  the  FY2008  budget 
to  replenish  low  supplies  resulting  from  demands  in  Iraq  and  Afghanistan.  Reports 
from  November  2005  claimed  that,  because  of  increased  use,  ammunition  for  the  M2 
produced  in  1945  was  being  pulled  out  of  the  stockpile  despite  ammunition  produc¬ 
tion  increasing  five  times  in  size  from  2001  to  2005.  The  2008  budget  request  asked 


18  Department  of  the  Army,  FY2013  Budget  Estimates  Justification  Book,  Procurement  of  W&TCV,  February 
2012c. 
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for  a  restart  to  the  program  citing  a  “critical  shortage  of  serviceable  machine  guns  for 
our  Soldiers  who  are  participating  in  Operation  Enduring  Freedom,”  and  noted  “this 
request  is  based  on  deficiencies  identified  from  on-going  combat  operations  in  Afghan¬ 
istan  and  Iraq.”19 

The  fact  that  the  “new  start”  of  the  Army  procurement  could  build  off  continued 
production  for  non-U.S.  Army  consumers  of  the  weapon  provided  some  cost  savings, 
because  it  meant  that  the  Army  did  not  have  to  invest  in  rebuilding,  repairing,  or  mod¬ 
ernizing  production  technology  or  the  industrial  base.  However,  the  program  has  still 
faced  some  design  and  safety  issues  since  procurement  resumed  in  2007.  Specifically, 
in  the  FY2012  NDAA,  the  Army  revised  its  initial  budget  request,  allocating  money 
from  the  purchase  of  new  M2  .50  caliber  machine  guns  to  the  modification  of  existing 
weapons  to  enhance  and  fix  safety  issues  associated  with  the  weapon.  In  the  request 
submitted  to  Congress,  the  Army  stated  that  the  modification  and  repair  was  a  higher 
priority  than  the  purchase  of  new  weapons  and  SASC  agreed.20  The  resulting  drop  in 
quantity  procured  does  not  appear  to  have  affected  projected  unit  cost.  Funding  was 
requested  in  2013,  likely  because  wars  in  Afghanistan  and  Iraq  have  slowed,  resulting 
in  reduced  demand. 

On  the  whole,  the  performance  of  the  program  since  2008  has  shown  no  seri¬ 
ous  problems.  Figure  3.6  provides  little  evidence  to  suggest  that  there  was  a  significant 
cost  to  the  Army  associated  with  restarting  the  program  in  2008  or  that  the  program’s 
unit  cost  benefited  from  learning  or  from  new  production  efficiencies  over  the  past 
five  years.  Congressional  testimony  on  the  program  similarly  reveals  no  serious  cause 
for  concern,  with  the  exception  of  the  safety  issue  raised  in  the  FY2012  NDAA.  These 
trends  are  not  surprising,  given  the  fact  that  M2  has  been  in  production  for  approxi¬ 
mately  90  years  with  the  same  basic  design.  The  M2  is  a  fully  mature  weapon  line. 
With  countries  and  organizations  all  over  the  world  using  the  weapon,  it  is  consistently 
in  high  demand.  Technical  issues  and  production  concerns  should  not  be  an  issue  for 
the  M2.  The  unit  cost  of  the  weapon  should  remain  constant  and  steady,  which  the 
recent  Army  procurement  corroborates.  However,  the  age  of  the  program  may  also 
pose  some  risks.  It  is  possible  that  additional  or  more  serious  safety  and  design  issues 
may  arise  in  the  near  term  (similar  to  those  reported  in  the  FY2012  NDAA)  that  may 
affect  the  unit  cost  and  viability  of  the  program  going  forward. 

Although  the  M2  program  performs  fairly  well,  there  is  a  clear  and  strong  rela¬ 
tionship  between  the  quantity  procured  and  unit  cost  for  this  program  as  was  true  for 
others — the  more  M2s  procured,  the  lower  the  unit  cost.  In  most  years,  substantial 
changes  in  quantity  result  in  large  changes  in  unit  cost.  For  instance,  between  2008 


19  Department  of  the  Army,  FY2008  Budget  Estimates  Justification  Book,  Procurement  of  W&TCV,  February 
2007. 

20  U.S.  Senate,  National  Defense  Authorization  Act  for  Fiscal  Year  2012:  Hearing  Before  the  Armed  Services  Com- 
mittee,  Washington,  D.C.,  U.S.  Government  Printing  Office,  2011a. 
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Figure  3.6 

Unit  Cost  and  Quantity,  M2  .50  Cal 
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and  2009,  quantity  purchased  almost  doubled  and  unit  cost  fell  by  about  11  percent. 
Quantity  procured  increased  by  56  percent  between  2009  and  2010  and  unit  cost  rose 
45  percent.  The  maturity  of  this  program  suggests  that  unit  cost  is  driven  largely  by 
production  quantity,  as  other  aspects  of  production  have  long  since  been  refined. 

Given  the  ease  with  which  the  program  was  resumed  in  2008,  the  decision  to 
cease  funding  for  the  program  in  FY2013  may  not  have  any  serious  implications.  How¬ 
ever,  the  ease  of  the  restart  and  lack  of  serious  changes  in  unit  cost  may  be  based  on 
international  demand  for  the  M2  .50.  If  demand  falls  internationally,  then  the  implica¬ 
tions  of  stopping/restarting  the  program  may  be  more  severe. 

Minor  Concerns 

C-37A,  Air  Force,  Aircraft  Procurement 

The  C-37A  is  a  long-range  executive  passenger  jet  that  provides  worldwide  air  trans¬ 
portation  for  the  vice  president,  cabinet  members,  congressional  delegations,  presiden¬ 
tial  emissaries,  and  other  high-ranking  dignitaries  of  the  United  States.  When  first 
acquired,  the  C-37A  jets  replaced  the  aging  C-137  fleet,  which  had  grown  costly  and 
unsafe  to  operate.  The  C-37A  is  produced  by  the  Gulfstream  Aerospace  Corporation 
and  was  originally  constructed  for  the  commercial  market. 

The  first  Air  Force  purchase  of  C-37A  aircraft  occurred  using  FY1997  funds.  The 
initial  contract  covered  the  purchase  of  commercial  off-the-shelf  (COTS),  long-range 
aircraft  as  well  as  necessary  special  modifications.  It  also  included  an  option  for  up  to 
four  additional  units  before  2003.  The  first  two  aircraft  were  delivered  by  September 
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1998.  The  contracting  process  used  for  these  first  two  aircraft  provided  significant  sav¬ 
ings  to  the  Air  Force.  According  to  testimony  delivered  in  1998: 

With  few  exceptions,  commercially  available  designs,  options,  practices,  and  pro¬ 
cesses  were  used  throughout  the  acquisition.  Acquisition  practices  were  simpli¬ 
fied  to  mirror  those  employed  in  the  airline  industry  with  no  significant  unique 
requirements  to  accommodate  the  needs  of  the  Air  Force.  . . .  The  marketplace  and 
the  economic  benefits  gained  from  high  volume  production  of  similar  items  drove 
the  price.  The  supportability  package  was  designed  to  take  full  advantage  of  the 
worldwide  Boeing,  Gulfstream,  and  airline  product  support  capability.  No  unique 
support  structure  was  created  to  accommodate  the  Air  Force’s  requirement.21 

Although  the  Air  Force  did  not  ask  for  additional  funds  in  FY1998  for  the  pur¬ 
chase  of  C-37A  aircraft,  HASC  recommended  using  savings  from  the  Air  Force’s  favor¬ 
able  competitive  negotiations  for  the  C-37A  in  the  previous  year  ($27.1  million  from 
the  FY1997  appropriation)  plus  an  additional  $6  million  to  purchase  one  additional 
aircraft  for  the  U.S.  Army.  The  Army  completed  this  purchase  by  trading  in  an  older 
C-20  aircraft,  already  under  lease  from  Gulfstream  Aerospace,  to  facilitate  the  pur¬ 
chase  and  equipping  of  this  new  C-37A.22 

FY1999  funds  procured  four  additional  C-37A  aircraft,  but  the  Air  Force  did  not 
allocate  additional  funds  to  the  C-37A  program  between  FY2000  and  FY2005.  How¬ 
ever,  it  did  execute  a  ten-year  lease  contract  for  20  C-37A  aircraft  under  Section  8133  of 
the  FY2000  NDAA.  The  lease  contract  was  officially  awarded  to  Gulfstream  in  March 
2002  as  a  ten-year  indefinite  delivery/indefinite  quantity  contract,  with  the  option  of 
purchase  at  the  end  of  the  10  years. 

In  FY2006  and  FY2009,  the  Air  Force  purchased  one  additional  C-37.  The 
FY2010  funding  request  purchased  one  C-37A  for  operational  support.  FY2011  funded 
the  purchase  of  two  additional  aircraft,  and  FY2012  will  fund  the  purchase  of  another 
three.  FY2010  funding  purchased  a  new  aircraft,  but  FY2011  and  2012  funds  pur¬ 
chased  leased  aircrafts  whose  terms  of  lease  (Section  8133  of  the  FY2000  NDAA)  were 
coming  to  an  end.  No  additional  funds  were  allocated  for  the  FY2013  budget. 

Figure  3.7  shows  unit  cost  and  quantity  over  the  life  of  the  C-37A  program.  Over¬ 
all,  the  program  performs  fairly  effectively,  although  there  are  some  minor  concerns, 
including  an  increase  in  unit  cost  through  2009  and  uncertainty  about  the  future  of 
leased  aircraft.  Unit  costs  did  decline  in  FY2011  and  FY2012,  and  no  additional  funds 
were  allocated  in  2013. 


21  U.S.  Senate,  Acquisition  and  Technology  Changes  in  Acquisition  Procedures:  Hearing  Before  the  Armed  Services 
Committee,  Washington,  D.C.,  U.S.  Government  Printing  Office,  1998. 

22  U.S.  Senate,  Appropriations  Committee  Report  on  the  Department  of  Defense  Appropriation  Bill  1998,  Washing¬ 
ton,  D.C.,  U.S.  Government  Printing  Office,  1997. 
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Figure  3.7 

Unit  Cost  and  Quantity,  C-37A 
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Sharp  changes  in  unit  cost  reflect  a  close  link  between  cost  and  quantity.  Because 
each  individual  aircraft  is  costly,  even  small  changes  in  quantity  have  large  cost  impli¬ 
cations.  Unit  cost  is  significantly  higher  in  FY2009  and  FY2010  than  in  previous  and 
later  years  in  which  quantity  acquired  was  significantly  higher.  Lower  unit  costs  in 
FY2011  and  FY2012  may  reflect  the  fact  that  the  purchased  aircraft  were  not  new 
but  were  already  under  lease.  The  leasing  strategy  was  intended  to  significantly  reduce 
upfront  procurement  costs,  both  during  the  lease  period  and  afterward,  when  the 
option  for  purchase  arose.23  The  use  of  commercial  technology  also  contributed  to 
lower  unit  costs  in  the  program’s  early  years,  but  its  benefits  in  later  years  were  few. 
Finally,  it  is  worth  noting  that  the  unit  costs  of  the  C-37A  are  likely  affected  through¬ 
out  its  program  life  by  commercial  purchases  of  the  aircraft. 

Assault  Breacher  Vehicle,  Army/Marine  Corps,  Tracked  Combat  Vehicles 

The  Assault  Breacher  Vehicle  (ABV)  is  a  joint  Marine  Corps  and  Army  tracked  vehicle 
used  for  mine  clearing.  Although  both  services  procure  these  vehicles,  the  Marine 
Corps  was  the  developmental  lead.  The  program  has  changed  names  several  times, 
known  as  the  Combat  Breacher  Vehicle  until  FY2005  but  as  the  Assault  Breacher 
Vehicle  since  then.  The  ABV  largely  uses  COTS  and  nondevelopmental  items  (NDI), 
such  as  an  M1A1  Abrams  tank  chassis.  The  Army  describes  the  ABV  as  “a  tracked 
combat  engineer  vehicle  for  the  Marine  Air  Ground  Task  Force  and  the  Army  Fleavy 


23  General  Dynamics,  U.S.  Government  Leases  Five  General  Dynamics  Gulfstream  C-37A  Aircraft  in  $477 Million 
Lease  and  Maintenance  Agreement,  press  release,  Falls  Church,  Va.,  October  26,  2000. 


Oversight  of  ACAT  II  Programs  49 


Brigade  Combat  Team  (HBCT).  It  is  designed  to  breach  minefields  and  barrier  obsta¬ 
cles  to  enable  the  tanks  and  infantry  of  the  HBCT  to  maintain  pace  in  offensive 
combat  operations.  The  ABY  provides  crew  protection  and  vehicle  survivability  equal 
to  the  M1A1  Abrams  tank  hull  and  has  the  speed  and  mobility  to  keep  pace  with  the 
heavy  maneuver  force.”24 

The  ABY  was  designed  to  support  other  Marine  Corps  armored  vehicles.  Six  pro¬ 
totype  vehicles  were  built  for  testing  and  evaluation  from  2002  to  2006,  after  which 
the  ABV  received  a  MS  III  decision,  leading  to  full-rate  production  (FRP).  The  first 
new  vehicle  was  procured  by  the  Marine  Corps  in  FY2005,  along  with  three  refur¬ 
bished  prototypes.  The  use  of  refurbished  prototypes  was  intended  to  accelerate  ABV 
deliveries  to  operating  forces  in  support  of  Operation  Iraqi  Freedom  (OIF)  mission 
requirements.  In  FY2006,  the  program  was  consolidated  under  the  Explosive  Ord¬ 
nance  Disposal  Systems  program.  In  FY2006,  funding  procured  18  additional  vehicles 
for  the  Marine  Corps.  From  FY2008  through  FY2011,  the  Marine  Corps  procured  a 
total  of  23  additional  ABV.  Starting  in  FY2011,  Marine  Corps  appropriations  for  the 
ABV  program  have  gone  to  modernization  as  a  result  of  lessons  learned  in  Operation 
Enduring  Freedom  (OEF)  but  not  to  the  purchase  of  new  vehicles.  These  moderniza¬ 
tions  include  ancillary  equipment  including  plows,  blades,  and  front-end  equipment 
required  to  support  operational  units  in  dwell  and  upgrades  that  will  increase  surviv¬ 
ability,  including  locating  and  engaging  targets  at  night  and  increased  accuracy  during 
fire  and  maneuver.  Reports  by  the  Marine  Corps  suggest  that  these  upgrades  were  a 
high  priority.25 

Despite  some  hiccups  in  the  development  stages,  the  Marine  Corps  experienced 
relatively  stable  unit  costs  right  from  the  start  of  the  ABV  program,  even  as  its  pro¬ 
curement  quantities  varied  somewhat  over  the  period  from  FY2005  to  FY2011,  shown 
in  Figure  3.8.  Variation  in  procurement  quantities  is  associated  largely  with  demand 
from  contingency  operations  in  Iraq  and  Afghanistan.  As  noted  above,  appropriations 
for  FY  2011  and  the  next  fiscal  year  are  being  used  to  upgrade  existing  vehicles  rather 
than  to  procure  new  ones. 

The  Army  chose  the  ABV  to  support  the  armored  vehicle  needs  of  the  Active, 
Reserve,  and  National  Guard  components,  procuring  its  first  vehicles  in  FY2008.  In 
its  budget  justifications,  the  Army  noted  that  the  ABV  would  be  used  in  HBCTs 
and  that  “the  Army  currently  does  not  have  the  capabilities  provided  by  the  ABV.”26 
FY2008  and  FY2009  funds  acquired  a  total  of  21  ABVs.  FY2010  funds  procured  13 
new  vehicles.  The  Army  procured  19  in  FY2011  and  plans  to  procure  18  in  FY2012 
and  an  additional  61  over  the  period  FY2013  to  FY2017.  The  vehicle  has  been  so  suc¬ 
cessful  that  the  Seapower  and  Expeditionary  Forces  Subcommittee  FY2011  National 


24  Department  of  the  Army,  2012c. 

25  Despite  being  described  as  modernization,  these  funding  requests  are  included  in  the  P-1,  not  the  P-1M. 

26  Department  of  the  Army,  FY201 0  Budget  Estimates  Justification  Book,  W&TCV  Procurement,  February  2009. 
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Figure  3.8 

Unit  Cost  and  Quantity,  Assault  Breacher  Vehicle 
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Defense  Authorization  Act  Markup  in  HASC  recommended  an  increase  in  funding  to 
procure  the  additional  vehicles  noting  that  “Specifically,  the  mark  would  authorize  an 
additional  33  million  for  seven  Assault  Breacher  Vehicles,  which  are  instrumental  in 
mine  field  clearance.”27 

The  Army’s  ABV  program  has  manifested  somewhat  more  instability  and  cost 
growth  than  the  Marine  Corps  program,  especially  in  recent  years.  For  example,  unit 
costs  increased  almost  30  percent  between  FY2009  and  FY2010  despite  an  increase 
in  quantity  procured.  Cost  projections  after  FY2013  also  suggest  some  initial  cost 
growth,  including  an  increase  of  almost  25  percent  between  FY2014  and  FY2016.  This 
increase  can  be  explained  by  falling  procurement  quantities  and  demand  as  operations 
in  Iraq  and  Afghanistan  draw  to  an  end.  The  decision  by  the  Marine  Corps  to  delay 
the  purchase  of  additional  vehicles  may  also  contribute  to  recent  and  projected  units 
cost  increases  observed  in  the  Army’s  ABV  program.  There  is  some  indication  of  con¬ 
gressional  concern  about  the  health  of  the  Army  program  during  this  period  as  well, 
specifically  in  the  FY2012  Senate  Appropriations  Committee  Defense  Bill  markup.  In 
its  report,  the  Senate  recommended  a  small  decrease  in  funding  to  the  program  of  $2.9 
million,  noting  that  the  Army’s  request  included  “Unjustified  growth  in  matrix  sup¬ 
port  and  engineering  change  proposals.”28  No  further  details  were  provided. 


27  U.S.  House  of  Representatives,  Markup  ofH.R.  5136:  Hearing  Before  the  Subcommittee  on  Seapower  and  Expe¬ 
ditionary  Forces,  Washington,  D.C.:  U.S.  Government  Printing  Office,  2010. 

28  U.S.  Senate,  Report  on  the  Department  of  Defense  Appropriations  Bill  2012,  Washington,  D.C.,  U.S.  Govern¬ 
ment  Printing  Office,  2011b. 
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Unit  cost  is  expected  to  fall  again  in  2017,  when  procurement  quantities  rise, 
although  this  projection,  like  others,  is  subject  to  future  demand  and  revision.  Because 
the  Army  plans  to  procure  a  large  number  of  additional  vehicles,  projected  increases  in 
unit  cost  may  be  cause  for  some  additional  investigation. 

In  comparing  the  Army  and  Marine  Corps  programs,  it  is  worth  noting  that  the 
Marine  Corps  has  consistently  had  lower  unit  costs  for  the  ABV  than  has  been  the 
case  for  the  Army,  despite  purchasing  smaller  quantities.  This  may  be  an  accounting 
issue,  as  each  service  reports  program  cost  slightly  differently.  (We  calculate  unit  costs 
for  the  ABV  using  only  acquired  hardware,  not  associated  peripheral  and  support  or 
other  services,  to  minimize  cross-service  differences).  However,  the  difference  in  unit 
cost  between  the  two  services  may  also  warrant  additional  investigation  to  determine  if 
the  discrepancy  reflects  differences  in  program  management,  acquisition  processes,  or 
equipment  use.  Congressional  testimony  and  the  reports  of  HASC,  SASC,  HAC,  and 
SAC  have  not  identified  major  issues  or  slippage  in  development  or  delivery  time  lines. 

Hellfire  Missile,  Army,  Navy,  and  Air  Force,  Weapons/Missile  Procurement 

According  to  the  Air  Force’s  program  justification,  the  Predator  Hellfire  “is  an  air-to- 
ground  missile  system  that  provides  heavy  armor,  precision-kill  and  anti-personnel 
capability.  Laser  Hellfire  uses  semi- active  laser  terminal  guidance.  The  latest  variant 
provides  for  point  target  precision  strike,  defeats  future  advanced  armor  threat  and  is 
effective  against  countermeasures.”29  The  Air  Force  uses  Predator  Hellfire  missiles  for 
test,  training,  and  operations,  particularly  aerial  drone  strikes  by  aircraft  such  as  those 
conducted  by  the  MQ-1. 

The  original  Hellfire  missile  system  began  in  1974  as  an  Army  program.  Each 
variant  of  the  Hellfire  missile  is  best  suited  for  specific  types  of  environments  and  types 
of  attacks.  The  most  common  are  the  AGM-114K  and  AGM-114M.30 

The  Air  Force  Hellfire  missiles  are  procured  off-the-shelf  from  the  Army’s  Red¬ 
stone  Arsenal  or  the  Navy,  with  contractor  and  location  determined  by  lead  service 
contract.  In  each  fiscal  year,  quantities  are  based  on  current  estimated  price  for  pur¬ 
chase  through  the  Army  or  Navy.  This  means  that  unit  cost  varies  depending  on  lead 
service  or  foreign  military  sales  procurement  quantities.  In  most  years,  the  purchase 
occurred  through  the  Army.  Funding  requests  after  FY2008  included  Overseas  Con¬ 
tingency  Operation  (OCO)  funding  requests.  It  is  also  worth  noting  that  many  differ¬ 
ent  types  of  Hellfire  missiles  are  all  included  in  the  potential  purchases. 

The  Army  and  the  Navy  also  use  the  Hellfire  missile.  Procurement  by  both 
departments  over  the  1990s  and  early  2000s  was  limited  to  inventory  maintenance. 
The  Army  purchased  1,800  in  1997  and  368  in  1998,  and  the  Navy  procured  248  in 


29  Department  of  the  Air  Force,  FY2006  Budget  Estimates  Justification  Book,  Missile  Procurement,  February 
2005. 

30  Jane’s  Electro-Optic  Systems,  Hellfire  Systems  AGM-114  Hellfire/Hellfire  II,  January  19,  2011. 
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2001.  The  Navy  budget  justification  noted  that  this  purchase  was  intended  to  maintain 
inventory  levels.  Starting  in  2007,  however,  the  Air  Force’s  use  of  the  missile  and  opera¬ 
tions  in  OEF  and  OIF  significantly  depleted  inventories  of  the  Hellfire  missile.  Both 
departments  began  new  procurement  at  this  point.  Army  appropriations  have  primar¬ 
ily  purchased  the  Longbow  Hellfire  and  the  new  P+  variant,  both  with  higher  unit 
costs.  Navy  procurement  has  purchased  more  of  the  114K  and  114M  models,  similar 
to  those  procured  by  the  Air  Force. 

Figures  3.9  and  3.10  show  the  performance  of  the  Army  and  Navy  Hellfire  pro¬ 
grams.  In  general,  both  programs  appear  stable  over  the  period  under  consideration. 
Unit  cost  is  associated  with  quantity  in  both  cases.  The  Navy  program  has  been  the 
more  stable  (and  we  categorize  it  as  a  program  with  no  serious  problems).  The  graph 
shows  stability  in  unit  cost  despite  some  significant  changes  in  quantity  procured. 
This  stability  likely  reflects  the  maturity  of  the  program.  Navy  unit  costs  have  also 
been  more  stable  than  those  of  the  Army  (Figure  3.10)  or  the  Air  Force  (Figure  3.11, 
described  below).  This  may  reflect  slightly  higher  procurement  quantities  or  different 
missile  variants.  However,  it  is  also  worth  noting  the  drop  in  quantity  procured  pro¬ 
jected  in  the  out-years  for  this  system.  This  may  reflect  the  effect  of  weakening  demand 
for  new  missiles  as  the  wars  in  Iraq  and  Afghanistan  come  to  an  end.  As  will  be  noted 
below,  this  drop  in  procurement  of  the  Hellfire  is  consistent  across  services. 

The  Army’s  unit  costs  have  been  somewhat  higher  and  show  more  significant 
evidence  of  cost  growth  after  2008  than  either  the  Navy’s  or  the  Air  Force’s.  The 
Army  also  experienced  more  dramatic  changes  in  quantities  procured  over  the  time 
period  considered.  Initially,  unit  cost  fell  sharply  with  rising  procurement  quantities. 

Figure  3.9 

Unit  Cost  and  Quantity,  Hellfire,  Army 
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Figure  3.10 

Unit  Cost  and  Quantity,  Hellfire,  Navy 
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Between  FY2009  and  FY2010,  however,  quantity  purchased  fell  by  about  26  percent, 
and  unit  cost  rose  about  20  percent.  Although  a  sharp  drop  in  quantity  procured 
between  FY2011  and  FY2012  did  not  result  in  a  large  change  in  unit  cost,  a  further 
reduction  in  quantity  procured  projected  in  FY2013  does  lead  to  a  51  percent  increase 
in  unit  cost.  Some  increase  in  unit  cost  in  recent  years  may  reflect  the  procurement 
of  new,  more  advanced  Hellfire  variants.  Still,  the  rapid  cost  growth  in  the  Army’s 
Hellfire  program  since  2009  and  projected  in  the  out-years  is  some  cause  for  concern. 
Finally,  the  significant  differences  in  unit  costs  across  military  departments  may  war¬ 
rant  additional  investigation.  Differences  may  reflect  nothing  more  than  accounting 
differences  or  different  missile  variants,  but  they  may  also  be  evidence  of  differences  in 
acquisition  process,  contracting,  or  efficiency. 

Figures  3.11  and  3.12  show  that  Air  Force  procurement  of  the  Predator  Hellfire 
has  been  affected  by  a  significant  amount  of  instability,  which  is  somewhat  surprising, 
since  it  has  been  procuring  these  missiles  from  the  Army  and  Navy.  In  the  initial  years 
of  the  Air  Force  program,  unit  costs  rose  despite  rising  procurement  quantities.  Since 
the  missiles  are  procured  off-the-shelf  from  the  Army,  it  is  unclear  what  drove  this 
unit  cost  increase.  However,  the  increase  was  fairly  short-lived.  Costs  returned  to  their 
original  level  by  FY2007  and  remained  steady  or  declined  through  2010.  Some  of  the 
decline  in  cost  between  FY2007  and  FY2010  may  reflect  the  effect  of  new  procurement 
by  the  Navy  and  the  Army.  However,  costs  began  to  rise  again  in  FY2011.  This  increase 
in  unit  cost  can  be  attributed  almost  entirely  to  changes  in  quantities  purchased.  Unit 
costs  calculated  for  years  after  FY2010  using  the  FY2010  quantity  (1,175)  show  con¬ 
stant  or  falling  unit  costs  over  the  entire  period. 
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Figure  3.11 

Unit  Cost  and  Quantity,  Hellfire,  Air  Force,  Years  with  Purchase 
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Figure  3.12 

Unit  Cost  and  Quantity,  Hellfire,  Air  Force,  All  Years 
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excess  supply  later  on  or  elevated  unit  costs  for  an  extended  period.  Because  the  major¬ 
ity  of  the  observed  unit  cost  increase  is  in  the  out-years,  it  is  always  possible  that  it  will 
not  materialize.  Since  the  program  history  does  not  reveal  significant  or  consistent 
increases  in  unit  cost  or  schedule  overruns  and  in  fact  shows  the  program  as  able  to 
overcome  instability  (as  it  experienced  in  early  years),  there  is  reason  to  be  optimistic. 

The  Air  Force  program  has  had  higher  unit  costs  and  greater  instability  than  the 
Navy’s  program  but  lower  unit  costs  and  more  stability  than  the  Army’s  program  (since 
the  Army  restarted  its  procurement  of  the  system  in  2007).  Although  outside  the  scope 
of  this  effort,  these  cross-service  discrepancies  are  worth  additional  investigation,  since 
they  may  yield  program  management  insights. 

MQ-8,  Navy,  Aircraft  Procurement 

The  MQ-8  (also  known  as  the  vertical  takeoff  and  landing  tactical  unmanned  aerial 
vehicle  [VTUAV]  or  Fire  Scout)  was  developed  and  manufactured  by  Northrop 
Grumman  to  provide  reconnaissance  and  targeting  support  for  ground,  air,  and  sea 
forces.  The  MQ-8  is  effectively  a  redesign  of  an  earlier  VTUAV  program,  known  as 
the  RQ-8A,  which  the  Navy  terminated  in  December  2001.  The  program  continued 
under  Army  leadership,  however,  and  the  RQ-8B  entered  low  rate  initial  production 
(LRIP)  and  passed  MS  III  in  2003. 

The  Navy  made  the  decision  to  resume  funding  for  the  VTUAV  to  meet  litto¬ 
ral  combat  ship  (LCS)  mission  requirements  in  FY2004,  after  it  was  determined  that 
the  LCS  would  be  the  first  platform  for  the  VTUAV.31  The  Navy  signed  a  contract 
with  Northrop  Grumman  for  the  development  of  the  nine  MQ-8B  Fire  Scout  (known 
as  the  RQ-8B,  until  redesignated  in  June  2005)  vehicles  to  be  completed  by  2009. 
According  to  the  Navy’s  budget  justification,  the  program  “was  designed  to  provide 
real-time  intelligence,  surveillance  and  reconnaissance  data  to  tactical  users  without 
the  use  of  manned  aircraft  or  reliance  on  limited  joint  theater  or  national  assets.”32  The 
Navy  program  received  significant  support  from  Congress.  In  2003,  Congress  also 
recommended  allocating  additional  funds  to  the  MQ-8  program  to  develop  an  opera¬ 
tional  testing  system  and  acquire  eight  aircraft  for  use  with  the  LCS.33  In  2005,  the 
House  Committee  recommended  additional  funding  for  the  program  in  the  markup 
for  the  FY2006  NDAA.34 


11  Jane’s  Unmanned  Aerial  Vehicles  and  Targets,  Northrop  Grumman  MQ-8  Fire  Scout,  June  21,  2011. 

32  Department  of  the  Navy,  FY2008  Budget  Estimates  Justification  Book,  Aircraft  Procurement,  February  2007. 

33  U.S.  Senate,  Appropriations  Committee  Report  on  the  Department  of  Defense  Appropriations  Bill  2004,  Wash¬ 
ington,  D.C.,  U.S.  Government  Printing  Office,  2003. 

34  U.S.  Senate,  National  Defense  Authorization  Act  2006 Report,  Washington,  D.C.,  U.S.  Government  Printing 
Office,  2005.  FY2004-FY2006  program  funds  are  referenced  in  testimony  but  do  not  appear  in  the  Navy  budget 
or  the  P-1.  The  first  record  of  spending  is  in  FY2007.  However,  the  total  aircraft  acquired  by  FY2009  matches  that 
reported  in  earlier  congressional  reports  (8-10). 
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The  MQ-8B  entered  LRIP  in  May  2007.  In  January  2008,  Northrop  Grumman 
initiated  a  new  phase  of  development  with  the  first  flight  test  using  a  test  and  training 
control  segment.  This  is  a  shelterized  version  of  the  exact  consoles  and  other  equipment 
being  integrated  into  LCSs  for  operational  use  of  the  MQ-8B  Fire  Scout.  The  Navy 
awarded  an  LRIP  2  contract  in  September  2008.  The  LRIP  3  contract  procured  three 
additional  MQ-8B  aircraft.  This  delivery  was  completed  in  2011.  The  technical  and 
operational  evaluation  (OPEVAL)  started  in  2009.  Because  of  delays  in  the  LCS  devel¬ 
opment  schedule,  the  Navy  conducted  sea  trials,  OPEVAL,  and  initial  deployment  on 
board  the  USS  Mclnerney,  an  Oliver  Hazard  Perry— class  frigate. 

Starting  in  FY2012,  MQ-8  funds  began  to  procure  the  longer  range  MQ-8C 
vehicles.  This  upgrade  addressed  congressional  concerns  about  the  MQ-8B’s  limited 
range  and  reluctance  to  purchase  additional  MQ-8B  vehicles  so  close  to  the  comple¬ 
tion  of  the  upgraded  platform.35  A  minimum  of  28  MQ-8C  endurance  upgrade  air¬ 
craft  are  being  procured  between  FY2012  and  FY2015  to  support  an  Africa  Command 
Joint  Urgent  Operational  Needs  Statement  Rapid  Deployment  Capability.  Finally,  the 
U.S.  Army  had  selected  the  Fire  Scout  MQ-8  as  their  Class  IV  unmanned  aerial  vehicle 
(UAV)  for  the  Future  Combat  System  (FCS).  Coordination  with  the  U.  S.  Army  FCS 
program  intended  to  investigate  the  potential  cost  savings  for  both  programs  where 
system  commonalities  and  common  logistics  support  can  be  identified.  However,  the 
FCS  program  was  cancelled. 

Figure  3.13  shows  that,  as  with  other  programs,  there  is  a  clear  link  between  unit 
cost  and  quantity  for  the  MQ-8  program  and  evidence  of  instability.  During  early 
years,  unit  cost  rose  with  quantity  produced.  This  rising  costs  suggests  some  inefficien¬ 
cies  and  delays  associated  with  the  development  stages  of  the  system.  Unit  cost  will  hit 
a  high  point  in  2013  but  should  decline  steadily  afterward,  suggesting  potential  stabili¬ 
zation  with  program  maturity  able  to  cope  with  the  rapid  production  fluctuations.  The 
Army,  following  the  FCS  cancellation,  has  expressed  renewed  interest  in  the  program. 
This  potential  partnership  with  the  Army  and  the  large  purchase  for  Africa  Command 
should  keep  unit  costs  low. 

Although  the  program  does  not  seem  to  have  faced  any  severe  obstacles  or  per¬ 
manent  or  runaway  cost  increases,  it  did  face  initial  slippage  in  delivery  schedule  and 
struggled  with  program  starts  and  stops  along  the  way,  often  because  of  weak  service 
commitment  and  a  slow  development  time  line.36  This  instability  in  demand  and  devel¬ 
opment  issues  associated  with  the  move  to  the  more  advanced  MQ-8C  may  explain  the 
gradual  increase  in  unit  cost  over  the  2010  to  2013  period.  A  review  of  this  program 
also  highlights  again  the  potential  value  of  cross-service  partnerships  and  the  challenge 
of  building  sufficient  inventories  of  a  given  system  while  more  advanced  models  are 


35  U.S.  House  of  Representatives,  Conference  Report  on  Military  Construction  and  Veterans  Affairs  and  Related 
Agencies  Appropriations  Act  2012,  Washington,  D.C.,  U.S.  Government  Printing  Office,  2011. 

Jane’s  Unmanned  Aerial  Vehicles  and  Targets,  2011. 
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Figure  3.13 

Unit  Cost  and  Quantity,  MQ-8 
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being  developed.  Although  purchase  quantities  must  be  kept  high  enough  to  maintain 
operational  sufficiency  and  to  keep  the  unit  costs  of  the  program  from  rising  too  high, 
it  also  does  not  make  sense  to  invest  in  aging  systems  when  the  release  of  new  ones  is 
right  around  the  corner. 

RQ-11,  Army,  Aircraft  Procurement 

The  RQ-11  Raven  is  a  small,  unmanned  aircraft  system  that  can  handle  a  wide  vari¬ 
ety  of  intelligence,  surveillance,  and  reconnaissance  tasks  at  battalion  level  and  below. 
The  small  unmanned  aircraft  system  (SUAS)  is  hand-launched  and  provides  aerial 
observation,  day  or  night,  at  line-of-sight  ranges  up  to  10  kilometers.  Also,  the  air¬ 
craft  has  an  endurance  rate  of  90  minutes  and  can  deliver  color  or  infrared  imagery 
in  real  time  to  the  ground  control  and  remote  viewing  stations.  The  current  RQ-11 
Raven  program  originated  as  the  FQM-151  Pointer  UAV,  developed  by  AeroVironment 
for  the  Army’s  Military  Operations  in  Urban  Terrain  Advanced  Concept  Technology 
Demonstration  (ACTD)  program.  As  a  result  of  lessons  learned  during  this  program, 
the  Army  asked  AeroVironment  to  develop  a  smaller  system  that  would  be  more  man- 
portable.  A  proof- of- concept  vehicle  of  this  smaller  system,  named  the  “Flashlight” 
small  unmanned  aerial  vehicle,  first  flew  in  October  2001.  The  Flashlight  was  further 
developed  into  the  Raven  in  2002  under  the  Army’s  Pathfinder  ACTD  program.  How¬ 
ever,  the  Raven  prototype  was  hand-built  and  not  suitable  for  mass  production. 

The  first  LRIP  version  was  the  modified  Block  1  Raven  SUAS,  first  delivered  in 
May  2003.  Testing  of  the  Block  1  UAVs  revealed  some  drawbacks,  including  a  dif¬ 
ficult  launch  procedure  and  insufficient  flight  stability.  The  program  was  accelerated 
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in  2003  to  meet  the  needs  of  forces  deployed  to  OIF  and  OEF.  The  initial  SUAS  pur¬ 
chases  were  made  by  the  Army  under  the  Rapid  Equipping  Soldier  Support  Equip¬ 
ment  program  in  FY2003/04  under  an  urgent  wartime  requirement  to  support  forces 
deployed  in  OIF/OEF.  The  Army  initially  procured  185  SUAS  systems  in  FY2003/04 
(under  supplemental  funding  approved  in  FY2004).  In  FY2005,  the  Army  procured 
270  additional  systems  to  support  fielding  to  modular  units.  Each  “system”  contains 
three  Raven  aircraft. 

In  testimony  to  Congress  in  2004,  Army  leaders  described  the  initial  procure¬ 
ment  contracts  as  an  “acquisition  success”  in  which  the  Army  responded  flexibly  to  the 
demands  of  new  contingencies.  General  Thurman,  Director  of  the  Army  Aviation  task 
force  noted,  “in  a  little  more  than  one  year,  the  Army  UAV  project  manager,  in  concert 
with  the  rapid  equipping  force  and  Natick  labs,  put  together  a  proposal  for  an  initial 
buy  of  five  small  fixed  wing,  battery  powered  UAV  systems.37  As  a  result,  the  Raven 
has  become  the  small  UAV  of  choice  for  the  U.S.  Air  Force  and  the  Special  Operations 
Command.  The  Army  has  the  lead  for  a  joint  Service  small  UAV  working  group  that 
has  already  realized  benefits  in  payload  compatibility  and  communications.” 

The  SUAS  program  completed  MS  C  on  October  6,  2005,  and  successfully  com¬ 
pleted  initial  operational  test  and  evaluation  in  June  2006.  The  program  obtained  FRP 
authority  on  October  5,  2006.  FY2007  funding  (total  includes  $5.4  million  received  in 
GWOT  supplemental)  procured  20  small  systems  hardware,  contractor  logistics  sup¬ 
port,  and  new  equipment  training  each  year.  FY2008  and  FY2009  funds  procured  100 
and  168  SUAS  (officially  called  Raven  only  in  2010),  respectively,  program  manage¬ 
ment  support,  contractor  logistics  support,  and  new  equipment  training. 

The  Raven  system  was  further  improved  in  FY2009  with  the  addition  of  a  digital 
data  link  (DDL).  According  to  the  Army’s  budget  justification,  the  “DDL  enhance¬ 
ment  improved  operational  capability  for  the  Warfighter  by:  Increasing  the  number  of 
channels  that  can  be  selected  allowing  for  more  air  vehicles  to  be  flown  in  a  smaller 
area;  Improvement  in  operational  range  through  relay  capability;  Incorporating  encryp¬ 
tion  capability;  and  Integration  of  advanced  digital  payloads.”38  The  first  DDL  systems 
were  fielded  in  December  2009.  Program  budgets  for  FY2010  and  later  include  DDL 
in  the  production  baseline.  Additionally,  retrofit  kits  were  procured  to  bring  all  non- 
DDL-equipped  systems  to  the  DDL  configuration.39  Starting  in  FY2010,  the  program 
was  renamed,  from  SUAS,  covered  under  “other  procurement,”  to  “RQ-11  Raven”  (its 
unofficial  title  before  this),  covered  under  Army  aircraft  procurement. 


37  U.S.  House  of  Representatives,  Fiscal  Year  2005  Budget:  Flearing  Before  the  Subcommittee  on  Tactical  Air  and 
Land  Forces,  Washington,  D.C.,  U.S.  Government  Printing  Office,  2004. 

38  Department  of  the  Army,  FY2012  Budget  Estimates  Justification  Book,  Aircraft  Procurement,  February,  2011. 

39  Retrofit  kits  are  included  in  the  P-1  appropriations  but  are  not  included  in  the  unit  cost  calculations  for  the 
RQ-1,  discussed  below. 
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FY2010  base  funding  of  $35  million  procured  206  systems,  which  equates  to  618 
air  vehicles.40  FY2010  OCO  funding  of  $45  million  procured  258  SUASs,  and  472 
DDL  retrofit  kits.  Also  in  that  year,  HAC  recommended  a  reduction  in  funding  with 
the  rationale  that  these  funds  had  been  requested  ahead  of  need.  FY2011  base  funding 
of  $20.2  million  procured  312  air  vehicles,  and  FY2011  OCO  funding  of  $17.4  mil¬ 
lion  procured  248  DDL  retrofit  kits.  FY2012  base  funding  of  $70.8  million  procured 
424  systems  (1,272  air  vehicles)  and  covered  the  hardware  costs  of  ground  control  sta¬ 
tions,  remote  video  terminal,  initial  spares  package,  and  Vampire  license  and  Vampire 
installation  trainers.  FY2012  contract  award  date  was  moved  from  December  2011  to 
April  2012  to  incorporate  additional  OCO  funding  appropriated.41  This  funding  was 
requested  by  the  Army  for  “war  replacement  vehicles.”  Because  of  the  short  production 
runs,  this  change  does  not  affect  the  schedule  for  future  awards.  FY2013  base  funding 
of  $25.8  million  will  procure  78  SUAS  (234  air  vehicles  with  gimbal  payload  engineer¬ 
ing  change  proposal)  as  well  as  spares  and  other  peripheral  equipment. 

The  Marine  Corps  also  adopted  the  Raven  B  as  the  primary  UAV  in  its  Force 
Protection  Airborne  Surveillance  System  program.  It  procured  somewhat  fewer  Raven 
aircraft  than  the  Army  and  only  in  2008,  2010,  and  2013.  In  other  years,  it  purchased 
support  systems  and  DDL  links  for  its  existing  systems.  The  Marine  Corps  uses  the 
Raven  “to  provide  the  company/detachment  level  with  airborne  reconnaissance  to  aid 
in  detecting,  identifying,  and  engaging  or  avoiding  enemy  units.  The  UAS  air  vehicle 
autonomously  gathers  and  transmits  imagery  of  the  tactical  situation  in  near-real  time 
at  a  range  of  up  to  ten  kilometers.”42 

There  are  some  minor  concerns  about  the  RQ-11  program,  stemming  from  its 
unit  cost/quantity,  as  shown  in  Figure  3.14.  The  graph  shows  instability  early  in  the 
program  and  significant  fluctuation  in  quantity  procured.  Unit  cost  is  projected  to 
increase  slowly  but  steadily  between  FY2012  and  FY2016,  totaling  an  increase  of  61 
percent  over  this  time  period.  There  is  also  evidence  of  a  link  between  unit  cost  and 
quantity.  Changes  in  procurement  quantity  during  FY2005,  FY2008,  and  FY2010- 
2012  all  have  a  clear  corresponding  effect  on  unit  cost.  Some  of  the  change  in  unit  cost 
may  be  associated  with  technological  upgrades  that  made  the  system  more  expensive, 
starting  with  the  FY2010  or  FY2012  acquisition.  However,  this  does  not  explain  the 
continuing  increases  in  cost  after  FY2010  or  FY2012.  After  FY2013,  quantity  procured 
is  expected  to  stabilize  while  unit  cost  will  slowly  grow.  However,  these  projections 
do  not  provide  information  on  specific  costs  per  aircraft  (only  for  the  total  program) 


40  In  early  years,  the  purchase  of  aircraft  is  provided  by  air  vehicle,  but  in  later  years,  budget  requests  list 
unmanned  aerial  systems  (UAS)  of  varying  types,  each  with  different  numbers  of  air  vehicles.  The  unit  cost  cal¬ 
culations  continue  to  reflect  per  vehicle  cost. 

41  Department  of  the  Army,  FY2013  Budget  Estimates  Justification  Book,  Aircraft  Procurefnent,  February,  2012a. 

42  Department  of  the  Navy,  FY2013  Budget  Estimates  Justification  Book,  Marine  Corps  Procurement,  February 
2012a. 
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Figure  3.14 

Unit  Cost  and  Quantity,  RQ-11 
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and  may  be  revised.  The  observed  cost  increase  since  2010  and  the  general  instability 
in  procurement  quantities  are  areas  where  additional  investigation  may  be  warranted. 
Possible  explanations  for  the  observed  instability  include  the  accelerated  development 
time  line  resulting  from  operations  in  Iraq  and  Afghanistan,  ongoing  technological 
improvements  to  the  system,  challenges  associated  with  integrating  the  new  technol¬ 
ogy  into  military  operations,  and  the  newness  of  the  program  itself. 

Marine  Corps  procurement  of  the  RQ-11  has  been  similarly  inconsistent  and  has 
also  demonstrated  gradually  rising  unit  costs.  However,  since  there  are  only  three  data 
points  (2008,  2010,  and  projected  2013),  it  is  difficult  to  draw  any  real  conclusions  and 
we  do  not  include  a  graph  of  this  program. 

Congressional  testimony  does  not  raise  any  specific  concerns  about  the  program. 
Testimony  in  FY2005  and  FY2006  praised  the  program  for  the  speed  of  development 
and  the  flexibility  the  Army  demonstrated  in  accelerating  the  program  to  meet  the 
urgent  stated  needs  of  warfighters. 

Some  Concern 

MTVR,  Weapons  and  Tracked  Vehicles,  Marine  Corps 

According  to  the  budget  justification,  “the  MTVR  is  a  joint  U.S.  Army/U.S.  Marine 
Corps  program  to  replace  the  existing  medium  tactical  motor  transport  fleet  of  M809/ 
M939  series  trucks  with  cost-effective,  state-of-the-art  technology  improvements.” 
“Major  improvements  include  a  new  electronically  controlled  engine/transmission, 
independent  suspension,  central  tire  inflation,  antilock  brakes,  traction  control,  cor- 
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rosion  control,  and  safety/ergonomic  features.”  Prototype  and  development  contracts 
were  awarded  in  1996.  At  the  time,  the  Marine  Corps  justification  stated  that  the 
remanufacture  was  mandatory,  because  of  the  aging  of  existing  vehicles.43 

The  program  faced  some  hurdles  in  getting  initial  RDT&E  funds.  In  FY1995, 
HASC  denied  the  Marine  Corps  request  for  $1.5  million  to  begin  RDT&E  for  its 
replacement  program.44  This  reflected  the  committee’s  disagreement  with  the  Marine 
Corps  efforts  to  involve  the  Army  National  Guard  and  Army  Reserve  in  a  joint  pro¬ 
gram  because  this  would  “tend  to  undercut  the  Army’s  plans  for  replacing  its  fleet.  The 
committee  does  not  concur  with  the  Marine  Corps  request  of  $1,545  million.”  At  the 
same  time,  the  committee  also  directed  that  the  Secretary  of  Defense  report  to  the 
congressional  defense  committee  by  March  31,  1995,  on  measures  that  will  be  taken  to 
harmonize  the  services  requirements  and  replacement  plans. 

The  program  successfully  received  RDT&E  funding  in  FY1996,  as  a  joint  Army 
and  Marine  Corps  program,  although  the  Marine  Corps  took  the  development  lead 
during  the  program’s  early  years.  In  that  year,  $3.5  million  was  allocated  for  initiation 
of  a  MTVR.  Additional  RDT&E  funding  in  FY1998  ($9  million)  was  spent  to  pro¬ 
cure  prototypes  from  two  contractors,  and  $13.1  million  was  budgeted  in  FY1999  and 
FY2000  for  a  special  bodies  program,  which  will  be  an  option  item  on  the  production 
contract  to  build  and  test  the  wrecker,  dump,  and  tractor  variants  of  the  MTVR.  Out- 
year  funding  was  also  allocated  to  source  selection  board,  testing,  management,  etc. 
Procurement  funding  in  FY1999  began  the  LRIP  for  production  ramp-up  and  first 
article  testing.  Funding  was  also  provided  for  various  integrated  logistics  support  func¬ 
tions,  such  as  factory  training,  documentation,  and  support. 

The  production  contract  used  was  an  Army  multiyear  fixed  price  contract  with  an 
economic  price  adjustment.  The  multiyear  contract  provided  significant  costs  savings, 
estimated  in  FY1999  to  save  approximately  6.5  percent  of  total  funding.  According  to 
the  Marine  Corps  program  justification,  the  reasons  for  the  cost  savings  include  “fixed 
contractor  facilitization  costs  can  be  amortized  over  a  large  base;  the  work  force  is  sta¬ 
bilized  over  a  longer  period;  subcontractor  purchases  result  in  much  larger  volume  dis¬ 
counts;  ‘hard  tooling’  commitments  are  made  in  lieu  of ‘soft  tooling’  expediencies;  and 
line  changeover/setup  costs  are  reduced  or  eliminated,  to  name  but  a  few.”45  Although 
anticipated  contract  costs  and  anticipated  cost  avoidance  were  only  estimates,  the  esti¬ 
mates  used  by  Marine  Corps  were  based  on  substantiated  data  and  industry  participa- 


43  Department  of  the  Navy,  FY1998  Budget  Estimates  Justification  Book,  Marine  Corps  Procurement,  February 

1997.  Although  this  was  a  joint  program  with  the  Army,  Army  procurement  of  medium  tactical  vehicles  is  not 
considered  in  this  summary  because  it  qualified  as  an  MDAP. 

44  U.S.  House  of  Representatives,  Armed  Services  Committee  Report  on  the  National  Defense  Authorization  Act  for 
Fiscal  Year  1995,  Washington,  D.C.:  U.S.  Government  Printing  Office,  1994a. 

45  Department  of  the  Navy,  FY1999  Budget  Estimates  Justification  Book,  Marine  Corps  Procurement,  February 

1998. 
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tion  and  had  been  approved  by  MDA  and  corroborated  by  independent  cost  estimates 
by  FY2000. 

The  program  justibcation  also  emphasized  the  relatively  low  risk  associated  with 
the  program,  since  the  MTVR  program  was  a  replacement  effort  that  used  COTS 
and  NDI.  The  justibcation  states  that  “performance  specibcation  is  stable  and  based 
upon  proven  performance  and  design  parameters  demonstrated  in  the  Marine  Corps 
Advanced  Technology  Transition  Demonstrators  (MCATTDs).”  The  program  was 
considered  high  priority  by  both  Congress  and  Marine  Corps  Combat  Development 
Command  and  stable  funding  throughout  the  period  was  expected. 

New  vehicles  were  procured  in  each  year  between  FY1999  and  FY2003.  These 
included  lower-cost  basic  and  long-bed  cargo  variants,  as  well  as  more  expensive  dump, 
wrecker,  and  extra-long  wheel  base  (XLWB)  variants.46  The  higher  cost  was  driven  by 
technology  development  and  the  fact  that  the  dump,  wrecker,  and  XLWB  variants 
were  also  behind  the  basic  and  long-bed  cargo  variants  in  the  development  process. 
Whereas  the  basic  and  long-bed  cargo  variants  entered  FRP  in  FY2001,  LRIP  quan¬ 
tities  for  dump  and  wrecker  span  FY2002  through  FY2003  (dump  LRIP  total  is  47 
and  wrecker  LRIP  total  is  308).  In  addition,  “pre-production  for  the  more  expensive 
vehicles  will  be  bay-built  models  to  facilitize  the  contractor’s  assembly  line  and  prepare 
work  instructions  for  those  specibc  variants.  This  process  is  time  consuming  as  denoted 
in  delivery  schedule,  but  must  be  done  prior  to  incorporating  them  into  the  multi-year 
production  line.  Eventually  these  vehicles  will  be  brought  up  to  full  production  con- 
bguration  and  helded.”47 

FY2004  to  FY2010  funds  continued  to  purchase  a  mix  of  MTVR  vehicles  along 
with  the  support  systems  needed  to  operate  the  vehicles.  The  dump,  wrecker,  and 
XLWB  MTVR  variants  entered  FRP  in  FY2006.  Also  starting  in  FY2006,  funds  were 
allocated  to  purchase  armor  kits  to  protect  the  MTVRs  from  attacks  by  improvised 
explosive  devices.  Starting  in  FY2008,  new  vehicles  came  equipped  with  this  armor. 
By  FY2010,  all  MTVR  vehicles  were  armored.  Funds  in  FY2011,  FY2012,  and  FY2013 
were  allocated  to  both  baseline  and  OCO  procurement,  although  in  each  case,  the 
majority  of  funding  came  from  OCO  procurement.  The  MTVR  was  an  important 
vehicle  during  operations  in  Iraq  and  Afghanistan,  and  additional  vehicles  had  to  be 
acquired  to  replace  those  that  were  worn  out  during  these  two  contingencies. 

The  Marine  Corps  significantly  revised  its  request  in  FY2012,  from  $392.4  mil¬ 
lion  to  $98  million,  following  a  significant  reduction  in  the  acquisition  target  for  the 
program.  Although  there  is  no  clear  explanation  for  the  reduction,  it  may  have  been 
driven  by  the  Marine  Corps’  own  identification  of  other,  more  pressing  operational 


46  In  some  years,  service  budget  requests  include  the  data  necessary  to  break  the  unit  cost  out  into  program- 
specific  pieces.  We  could  do  this  if  helpful  and  valuable,  but  for  now,  we  have  simply  calculated  average  unit  cost. 

47  Department  of  the  Navy,  FY2003  Budget  Estimates  Justification  Book,  Marine  Corps  Procurement,  February 
2002a. 
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needs.  It  requested  that  the  savings  from  the  MTVR  reduction  be  reallocated  to  other 
Marine  Corps  priorities,  a  request  the  Senate  granted.48 

Figure  3.15  suggests  some  cause  for  concern  with  the  MTVR  program.  Specifi¬ 
cally,  the  graph  suggests  gradual  but  steady  growth  in  unit  cost  and  an  apparent  decline 
in  Marine  Corps  interest  in  the  program,  as  procurement  falls  to  zero.  In  early  years, 
unit  cost  fell  with  increasing  quantity  procured.  After  2002,  however,  costs  began 
to  rise  as  procurement  quantities  fell.  Some  instability  in  quantities  procured  can  be 
explained  by  the  start  of  military  operations  in  Afghanistan  and  then  Iraq.  However, 
even  after  procurement  quantities  stabilized  (in  about  2008),  unit  cost  continued  to 
rise.  Marine  Corps  budget  justifications  and  congressional  testimony  do  not  address 
this  cost  increase. 

Cost  growth  may  be  associated  with  a  shift  in  the  variants  of  MTVRs  procured. 
As  noted  above,  numerous  variants  of  the  MTVR  are  available  including  wrecker, 
tractor,  dump,  cargo,  cargo  XLWB,  as  well  as  armored  versions  of  all  the  preceding. 
Differing  variants  have  different  unit  costs.  Unfortunately,  Marine  Corps  budget  jus¬ 
tifications  do  not  contain  consistent  and  sufficient  visibility  into  variant  unit  costs  to 
definitively  attribute  cost  growth  to  lot  buys  containing  a  higher  ratio  of  expensive 
variants.  However,  limited  glimpses  into  varying  unit  costs  lend  credence  to  this  idea. 
In  the  FY2012  Marine  Corps  Procurement  Budget  Justification  Book,  six  MTVR  vari¬ 
ants  are  differentiated  in  the  MTVR  Exhibit  P-5,  which  presents  cost  analysis  informa- 

Figure  3.15 

Unit  Cost  and  Quantity,  MTVR 
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48  U.S.  Senate,  Report  on  the  National  Defense  Authorization  Act  for  Fiscal  Year  2012,  Washington,  D.C.:  U.S. 
Government  Printing  Office,  2012. 
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tion  (see  Department  of  the  Navy,  2012c).  The  FY2010  appropriation  procured  vari¬ 
ants  with  unit  costs  that  ranged  from  $226,000  for  the  cargo  version  to  $609,000  for 
the  wrecker  variant. 

Also  noteworthy  is  the  sharp  reduction  in  acquisition  targets  for  the  MTVR  pro¬ 
gram  in  the  FY2013  Defense  Appropriations  Bill  and  the  Marine  Corps  request  for  the 
reallocation  of  MTVR  funds  to  other  high-priority  needs.  This  may  reflect  a  response 
to  the  more  resource-constrained  environment  faced  by  DoD,  waning  interest  in  the 
MTVR  as  wars  in  Iraq  and  Afghanistan  draw  to  a  close,  or  some  combination  of  the 
two.  However,  it  raises  some  concern  about  the  future  viability  of  the  program.  The 
Army’s  participation  in  and  commitment  to  the  program  is  another  issue  that  may 
have  affected  program  performance.  Specifically,  Army  procurement  of  MTVR  vari¬ 
ants  qualified  as  an  MDAR  The  volume  of  this  purchase  and  the  Army’s  participation 
in  the  program  may  have  held  unit  cost  down  or  kept  it  more  stable  than  it  might  oth¬ 
erwise  have  been.  Congressional  reports  and  other  secondary  sources  do  not  report  any 
additional  major  problems  with  the  MTVR  program. 

If  the  performance  of  the  MTVR  program  continues  to  be  an  area  of  interest, 
future  analyses  might  assess  the  unit  cost  and  quantity  of  each  variant  of  MTVR  sepa¬ 
rately  to  tease  out  the  specific  drivers  of  cost  growth  more  carefully. 

RAM,  Navy,  Weapons  Procurement 

RAM  is  a  high-fire-power,  low-cost,  lightweight  ship  self-defense  system  used  to 
engage  antiship  missiles.  RAM  was  first  operational  in  November  1992,  constructed 
as  a  North  Atlantic  Treaty  Organization  collaborative  project  with  the  Federal  Repub¬ 
lic  of  Germany.  Approval  for  FRP  ,  given  at  MS  III,  was  granted  on  May  6,  1993.  In 
January  1998,  Raytheon  acquired  Hughes  Missile  Systems  Co.,  making  Raytheon  the 
U.S.  prime  contractor.49 

The  program  faced  some  challenges  in  its  development  phase.  In  1991,  the  HAC 
recommended  terminating  the  RAM  program  because  of  concerns  about  its  ability  to 
defeat  future  threats  and  the  level  of  commitment  within  the  Navy  for  the  program.50 
The  Navy  subsequently  initiated  a  review  of  the  program,  focused  on  improving  the 
system  and  eventually  deploying  it  on  additional  ships.  The  FY1992  appropriation 
included  $2.95  million  for  research  and  development  efforts  to  improve  the  RAM 
and  $5  million  for  an  infrared  upgrade  to  the  system.  At  this  time,  HAC  requested  a 
restructured  program  that  reflected  the  Navy’s  planned  upgrades  in  the  FY1993  budget 
request. 

In  April  1993,  the  decision  was  made  to  implement  a  block  upgrade  (the  initial 
RAM  is  known  as  Block  0).  The  upgraded  Block  1  missile  added  the  capability  of 


49  Department  of  the  Navy,  FY2006 Budget  Estimates  Justification  Book,  Weapons  Procurement,  February  2005. 

50  U.S.  House  of  Representatives,  Committee  on  Appropriations:  Report  on  the  Department  of  Defense  Appropria¬ 
tions  Bill,  1992,  Washington,  D.C.:  U.S.  Government  Printing  Office,  1991. 
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infrared  all-the-way  guidance  to  the  original  missile  technology.  The  FY1995  appro¬ 
priation  included  $5  million  recommended  by  HASC  to  support  additional  research 
on  a  dual-thrust  motor  upgrade.  This  effort  was  incorporated  into  the  Block  1  redesign. 
Approval  for  major  modification  (Block  1)  was  granted  in  the  MS  IV  decision  of  May 
6,  1994.  A  successful  preliminary  design  review  of  the  Block  1  IR  upgrade  was  con¬ 
ducted  in  September  1995. 

Although  purchases  of  the  Block  0  missile  continued  during  the  development 
of  the  Block  1  missile,  the  quantities  of  these  purchases  were  scaled  back  somewhat, 
reflecting  a  desire  within  Congress  and  the  Navy  not  to  spend  money  on  weapon  sys¬ 
tems  that  would  soon  be  outdated.  In  discussions  over  the  FY1994  budget,  for  exam¬ 
ple,  the  Navy  asked  for  $58.5  million  to  buy  240  RAMs,  but  HASC  recommended 
only  $43.9  for  180  missiles,  noting  that  “that  the  [immediate]  need  for  RAM  missiles 
should  be  tempered  by  minimizing  costly  retrofit  requirements  for  the  current  mis¬ 
sile”  once  the  improved  version  was  complete.51  Purchases  of  Block  0  missiles  were 
gradually  phased  out  over  the  period  between  FY1996  and  FY1999.  However,  concerns 
about  the  viability  and  efficacy  of  the  RAM  continued.  FY1998  funding  for  the  RAM 
was  authorized  only  after  the  Secretary  of  the  Navy  reported  on  the  Navy’s  ability  to 
address  close-in  defense  of  Navy  surface  ships  and  how  the  RAM  would  support  that 
effort. 

In  the  FY2000  appropriation  report,  HASC  noted  “unexplained  cost  growth  for 
component  improvement,  government  in-house  engineering,  and  production  accep¬ 
tance”  and  recommended  a  cut  in  the  Navy’s  request  by  $1  million,  another  sign  of 
potential  problems  in  the  program.52  Funds  that  year  procured  90  Block  1  missiles. 

Approval  for  FRP,  MS  III,  of  the  Block  1  missile  was  granted  on  January  20, 
2000.  The  United  States  and  the  Federal  Republic  of  Germany  signed  a  new  Block  1 
production  memorandum  of  agreement  (MOA)  on  December  18,  2001,  to  coopera¬ 
tively  produce  Block  1  missiles,  launchers,  and  ordnance  alterations  (ORDALTs)  with 
block  upgrades  for  older  missiles.  The  ORDALTs  are  included  in  P-1  budget  appropria¬ 
tion  requests  but  were  excluded  from  the  unit  cost  calculations  in  our  analysis. 

Combined  developmental  test/operational  test  began  in  June  2003  and  extended 
into  FY2004,  using  the  existing  self-defense  test  ship  to  ensure  operationally  realistic 
tests  for  determining  that  RAM  with  the  new  helicopter-air-surface  software  retained 
capability  against  antiship  cruise  missiles.  The  United  States  and  the  Federal  Repub¬ 
lic  of  Germany  continue  to  cooperatively  produce  Block  1  missiles,  launchers,  and 
ORDALTs  through  the  Block  1  production  MOA,  Amendment  2,  signed  September 
28,  2010. 


51  U.S.  House  of  Representatives,  National  Defense  Authorization  Act  for  Fiscal  Year  1994  Report,  Washington, 
D.C.:  U.S.  Government  Printing  Office,  1993. 

52  U.S.  House  of  Representatives,  National  Defense  Authorization  Act  for  Fiscal  Year  2000  Report,  Washington, 
D.C.:  U.S.  Government  Printing  Office,  1999b. 
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The  Block  2  missile  upgrade  is  intended  to  more  effectively  counter  maneuver- 
able  antiship  missiles.  The  U.S.  Navy  awarded  Raytheon  Missile  Systems  a  $105  mil¬ 
lion  Block  2  RAM  development  contract  on  May  8,  2007.  Missile  development  was 
complete  in  December  2010,  with  guided  test  flights  in  2011  and  LRIP  beginning  in 
late  2012.  Appropriations  for  the  RAM  purchased  only  Block  2  missiles  starting  with 
FY2010. 

The  quantity  of  missiles  purchased  has  been  fairly  consistent,  suggesting  contin¬ 
ued  commitment  to  the  program  over  time.  Appropriations  in  recent  years  have  gone 
to  fund  Block  2  missiles  as  well  as  ORDALTs. 

The  RAM  program  is  one  that  raises  some  more  significant  concerns  about  pro¬ 
gram  health  and  unit  cost  growth,  as  suggested  in  Figure  3.16.  First,  the  program 
experienced  some  gradual  cost  growth  over  the  first  eight  or  so  years  of  its  life,  through 
about  2003.  This  cost  growth  was  initially  associated  with  declining  procurement 
quantities  but  continued  even  after  procurement  stabilized.  Procurement  quantity  was 
mostly  stable  over  the  period  between  2003  and  2010,  although  unit  cost  crept  up 
almost  28  percent  during  this  period.  In  addition,  large  buys  of  ORDALTs  in  2006 
(120)  and  2008  (57)  caused  unit  cost  to  spike  while  quantity  remained  constant.  Fiow- 
ever,  projected  program  performance  after  2011  suggests  significantly  more  program 
performance  problems,  including  an  increase  in  quantity  and  a  sharp  increase  in  unit 
cost.  This  jump  in  unit  cost  can  be  explained  by  the  transition  of  RAM  procurement 
from  Block  1  to  Block  2.  In  addition,  once  quantity  returns  to  its  original  level  in 
FY2012,  costs  remain  significantly  higher  than  they  were  pre-2012.  Block  2  missiles 
are  significantly  more  expensive,  given  their  technological  advances.  It  may  also  be 

Figure  3.16 
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that  falling  demand  from  other  consumers,  such  as  Germany,  has  affected  unit  cost. 
Although  congressional  testimony  and  budget  markups  do  not  specifically  flag  this 
cost  growth,  it  is  also  not  explained  in  Navy  budget  justifications. 

The  large  increase  in  unit  cost  for  the  new  missile,  general  instability,  unit  costs 
that  have  gradually  but  consistently  risen  over  time,  and  an  unclear  relationship 
between  procurement  quantity  and  unit  cost  suggest  some  reason  for  concern  about 
RAM  program  health.  These  issues  warrant  further  investigation. 

Summary 

Overall,  the  ACAT  II  programs  in  our  selected  set  perform  fairly  effectively.  Instabil¬ 
ity  in  cost  and  quantity  are  more  significant  problems  for  the  ACAT  II  programs  in 
the  selected  set  than  any  runaway  cost  growth  or  major  slippage  in  schedule  and  per¬ 
formance.  We  can  make  a  number  of  general  assessments  about  the  performance  of 
the  sample  ACAT  II  programs.  First,  across  programs,  there  is  a  strong  relationship 
between  unit  cost  and  annual  quantity  procured.  Unit  cost  rises  as  annual  quantity 
procured  declines.  For  some  programs  (the  healthiest  ones),  the  relationship  between 
cost  and  annual  quantity  is  fairly  stable  and  systematic  over  time.  This  is  true  of  the 
LCAC  SLEP  and  the  RAM  for  most  of  the  program  life.  However,  overall,  there  is 
too  much  variation  to  define  a  consistent  or  predictive  relationship  between  cost  and 
quantity  that  would  be  useful  to  policymakers.  As  noted  above,  some  of  the  observed 
instability  observed  in  several  of  our  programs  can  be  explained  by  changes  in  demand 
associated  with  the  end  of  contingency  operations  in  Iraq  and  Afghanistan. 

To  offer  a  more  quantitative  perspective  on  the  performance  of  our  ACAT  II 
set,  we  calculated  the  average  total  cost  growth  for  all  programs  over  the  period  from 
FY2010  to  FY2011.  We  found  that  average  total  cost  rises,  but  average  unit  cost  falls  an 
average  4.4  percent.  This  number  is  only  an  estimate  but,  as  will  be  discussed  below,  it 
is  at  least  as  good  as  the  performance  of  the  MDAPs. 

In  addition  to  the  relationship  between  unit  cost  and  quantity,  another  key  obser¬ 
vation  is  the  frequent  challenge  posed  by  program  instability.  As  noted  above,  insta¬ 
bility  in  cost  and  annual  quantity  procured,  not  runaway  cost  growth,  is  the  biggest 
challenge  faced  by  the  ACAT  II  programs  in  our  set  of  programs.  However,  although 
this  instability  is  somewhat  problematic,  it  does  not  cripple  the  programs  in  our  pro¬ 
gram  set.  Instability  is  often  associated  with  challenges  during  the  development  stages, 
driven  by  accelerated  time  lines,  failure  to  use  prototypes,  unstable  funding,  and  weak 
commitment  to  the  program  by  the  leading  military  department. 

Finally,  there  is  some  evidence  that  cost  growth  is  greater  among  more  costly 
ACAT  II  programs,  younger  programs,  and  programs  with  lower  total  procurement 
quantities.  Examples  and  the  relationship  between  program  size  and  cost  growth  will 
be  explored  in  more  detail  below,  after  the  assessment  of  MDAP  programs. 

Two  contracting  strategies  are  used  in  several  cases  and  appear  to  be  associated 
with  program  success  and  sometimes  program  stability.  These  are  the  use  of  partner- 
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ships  and  the  purchase  of  off-the-shelf  technology.  Cross-service  and  cross-national 
partnerships  support  success  when  they  spread  development  costs  and  increase  total 
production  quantities,  keeping  unit  costs  lower.  Partnerships  may  also  allow  for  pro¬ 
gram  starts  and  stops  without  normally  associated  “penalties,”  such  as  the  costs  to 
rebuild  the  industrial  base.  However,  partnerships  also  come  with  risk,  as  each  partner 
is  vulnerable  to  changes  in  the  demand  by  the  other.  The  ABV  is  a  good  example  of 
a  partnership  between  the  Army  and  Marine  Corps  that  had  both  costs  and  benefits. 
It  spread  development  costs  and  contributed  to  stability  in  unit  costs  for  the  Marine 
Corps.  The  Army  also  benefited  from  having  the  Marine  Corps  be  the  lead  service 
and  from  reduced  unit  costs  but  has  experienced  more  significant  program  instability, 
especially  as  demand  from  the  Marine  Corps  has  declined.  Partnerships  also  affected 
the  performance  of  the  M2  .50  caliber,  allowing  for  program  stability  despite  starts 
and  stops.  The  Hellhre  missile,  used  by  all  services,  also  reaped  the  benefits  of  cross¬ 
service  partnerships.  The  purchase  of  off-the-shelf  technology  has  also  supported  suc¬ 
cess  for  several  of  our  programs,  including  the  Hellhre  missile  and  the  C-37A.  This  is 
largely  because  of  reduced  development  costs  and  shorter  time  lines.  However,  cost 
savings  and  stability  benefits  are  smaller  than  expected  in  some  instances,  including 
the  C-37A. 

Although  most  ACAT  II  programs  perform  fairly  well,  some  seem  to  experience 
more  problems  than  others.  Some  programs  in  the  selected  set  are  affected  by  such 
problems  as  rebaselining,  development  delays,  incomplete  or  incorrect  program  docu¬ 
mentation,  contractor  incentive  issues,  and  program  execution  issues.  Although  cost/ 
schedule/quantity  revisions  exist,  they  are  still  not  crippling  in  every  case.  In  fact,  of 
the  three,  only  the  Army’s  program  has  been  cancelled,  because  of  low  priority.  The 
Evolved  SeaSparrow  Missile  program  is  currently  performing  fairly  well,  and  the  ver¬ 
dict  is  still  out  on  the  large  aircraft  infrared  countermeasures  (LAIRCMs). 

To  conclude  the  assessment  of  ACAT  II  programs,  it  is  worth  returning  to  our 
original  questions  regarding  ACAT  performance.  First,  do  ACAT  II  programs  face 
cost  and  schedule  issues  and,  if  so,  what  are  they?  The  answer  appears  to  be  yes,  but 
these  problems  are  not  permanent  or  crippling.  We  observe  some  instability  in  unit 
cost  and  annual  quantity  procured,  especially  early  in  the  program  life  cycle,  but  sta¬ 
bilization  is  common  and  likely  over  time.  We  did  not  find  evidence  of  runaway  cost 
increases  and  massive  schedule  slippage.  Our  overall  assessment  is  that  the  military 
departments  are  fairly  effective  at  managing  ACAT  II  programs.  Also  important  is  the 
observation  that  the  problems  ACAT  II  programs  do  face  are  fairly  diverse  and  pro¬ 
gram-specific  and  not  suggestive  of  a  single  serious  systemic  failure  on  the  part  of  the 
military  departments.  The  next  step  is  to  compare  the  performance  of  these  programs 
to  that  of  a  reasonably  comparable  set  of  MDAPs. 
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Comparison  of  MDAPs  and  ACAT  II  Programs 

As  noted  at  the  outset,  to  determine  if  additional  guidance  for  ACAT  II  programs  is 
needed,  we  need  to  evaluate  whether  there  are  problems  with  the  decentralized  over¬ 
sight  mechanisms  currently  used  to  manage  non-MDAPs  that  could  be  addressed  with 
centralized  reporting  and  control  guidelines.  One  way  to  make  such  an  assessment  is 
to  compare  the  performance  of  our  ACAT  II  set  with  a  set  of  MDAP  programs,  which 
already  benefit  from  (or  at  least  are  subject  to)  more  rigorous  reporting  and  control 
requirements.  To  conduct  such  a  comparison  we  pursued  two  lines  of  analysis.  First, 
we  reviewed  relevant  literature  on  MDAP  performance  to  identify  challenges  faced 
by  MDAPs  and  factors  that  tend  to  support  their  success.  This  includes  reports  by 
GAO  as  well  as  root  cause  and  cost  growth  analyses  conducted  by  RAND.  Second, 
we  directly  evaluate  the  performance  of  a  selected  set  of  MDAPs,  using  data  and  an 
approach  (described  in  detail  below)  that  are  identical  to  those  used  in  our  assessment 
of  ACAT  II  programs,  and  we  compare  these  results  with  our  selected  set  of  ACAT  II 
programs,  assessed  in  the  previous  section. 

Factors  Supporting  Success  and  Failure 

A  review  of  recent  GAO  assessments  of  MDAP  performance  identifies  common  factors 
that  predict  or  support  the  success  of  these  programs  as  well  as  challenges  and  prob¬ 
lems  they  often  encounter.  According  to  a  2010  GAO  report,  the  factors  that  contrib¬ 
ute  to  the  success  of  MDAPs  are  similar  to  those  that  support  the  success  of  ACAT  II 
programs  that  were  included  in  our  program  set.53  These  include  stable  funding,  use 
of  prototypes,  shorter  time  lines,  incremental  development  and  modest  development 
goals,  clear  and  well-defined  requirements,  use  of  developed  technologies,  the  forma¬ 
tion  of  realistic  cost  and  schedule  estimates,  and  use  of  independent  reviews  where 
possible. 

Turning  to  the  challenges  and  performance  issues  faced  by  MDAPs,  GAO  assess¬ 
ments  show  that,  once  again,  the  problems  that  plague  MDAPs  are  similar  in  nature  to 
those  faced  by  ACAT  IIs.  They  include  fluctuation  in  quantity,  costly  modernizations, 
upgrades,  production  inefficiencies,  accelerated  time  lines  without  prototypes,  slippage 
and  delays,  revisions  and  instability  in  program  demand  and  quantity  procured,  inac¬ 
curate  initial  cost  estimates,  and  missing  documentation.  However,  the  problems  faced 
by  MDAPs  and  ACAT  IIs  appear  to  diverge  in  scope,  severity,  and  duration.  First, 
MDAP  cost  growth  and  schedule  delays  are  more  extensive  and  severe  when  they 
occur.  It  is  worth  noting  that  one  reason  for  this  may  be  the  greater  complexity  of 
MDAP  programs,  which  makes  them  more  prone  to  development  and  other  technical 
delays  and  problems.  Although  there  is  no  way  to  account  fully  for  this  difference  in 


53  U.S.  Government  Accountability  Office,  Defense  Acquisition:  Strong  Leadership  Is  Key  to  Planning  and  Execut¬ 
ing  Stable  Weapons  Programs,  GAO-10-522,  Washington,  D.C.,  May  2010a. 
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our  analysis,  our  approach  of  selecting  the  largest  ACAT  II  programs  should  mitigate 
some  of  the  differences  in  complexity.  We  return  to  this  question  of  comparability 
below. 

Cost  Growth  and  Program  Size 

One  way  ACAT  IIs  and  MDAPs  may  be  different  is  in  the  relationship  between  pro¬ 
gram  size  and  cost  growth.  Several  previous  studies  have  reported  that  smaller  MDAP 
programs  tend  to  show  more  cost  growth.  For  example,  Drezner  et  al.  (1993)  and  a 
study  of  the  cost  estimation  methods  utilized  by  Ballistic  Missile  Defense  Organiza¬ 
tion54  both  report  evidence  of  such  a  relationship.55  A  2001  NAVAIR  study,  in  con¬ 
trast,  found  no  statistically  significant  difference  between  cost  growth  and  program 
size,  although  its  authors  report  that  the  variance  in  cost  growth  across  a  selected  set  of 
smaller  MDAPs  is  more  significant  (smaller  programs  are  more  likely  to  have  extremely 
high  or  extremely  low  cost  growth).56 

The  Drezner  et  al.  study  suggests  several  explanations  for  the  inverse  relation¬ 
ship  between  cost  growth  and  program  size.  First,  the  authors  observe  that  smaller 
programs  may  have  less  oversight,  leading  ultimately  to  more  severe  problems,  delays, 
and  cost  overruns.  Second,  it  may  simply  be  that  variance  in  unit  cost  is  more  visible  in 
percentage  terms  in  smaller  programs.  Third,  RDT&E  costs  make  up  a  larger  percent¬ 
age  of  total  cost  in  small  programs,  and  this  is  where  programs  are  more  susceptible  to 
cost  growth  and  schedule  delays.  Finally,  it  may  be  that  cost  growth  in  small  programs 
is  tolerated  because  it  is  smaller  in  size,  whereas  large  programs  that  experience  cost 
growth  in  early  stages  are  terminated. 

The  comparison  of  our  analysis  with  previous  work  comes  with  a  number  of 
important  caveats.  First,  past  work  considered  only  MDAP  programs  and  included  cost 
growth  resulting  from  growth  in  RDT&E  costs,  which  we  did  not  include  because  of 
limited  data  availability.  Second,  the  programs  assessed  by  previous  studies  were  not 
subject  to  the  same  reporting  and  control  requirements  as  today’s  MDAPs,  because  the 
majority  of  these  requirements  and  guidelines  came  into  effect  only  in  the  late  1980s 
and  1990s.  Third,  there  are  a  number  of  differences  between  the  fiscal  and  competi¬ 
tive  environment  now  and  that  examined  in  previous  work  (in  the  1980s  and  1990s) 
that  may  affect  this  relationship.  For  example,  there  has  been  significant  consolidation 
among  defense  contractors,  which  has  reduced  competition  and  affected  program  per- 


54  R.  L.  Coleman,  J.  R.  Summerville,  M.  DuBois,  and  B.  Myers,  Risk  in  Cost  Estimating:  General  Introduction 
&  BMDO  Approach,  paper  presented  at  the  33rd  Annual  DoD  Cost  Analysis  Symposium,  Williamsburg,  Va., 
February  2,  2000. 

55  Jeffrey  A.  Drezner,  Jeanne  M.  Jarvaise,  Ronald  Hess,  Paul  G.  Hough,  and  Daniel  M.  Norton,  An  Analysis  of 
Weapon  System  Cost  Growth,  Santa  Monica,  Calif.:  RAND  Corporation,  MR-291-AF,  1993. 

56  Capt.  Vince  Sipple,  Maj.  Edward  White,  and  Maj.  Michael  Greiner,  Surveying  Cost  Growth,  Washington, 
D.C.:  Office  of  the  Under  Secretary  of  Defense,  AT&L,  2004. 
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formance.  In  addition,  the  budget  situation  and  recent  wars  in  Iraq  and  Afghanistan 
have  altered  the  fiscal  environment  faced  by  both  contractors  and  DoD. 

Evaluating  MDAPs 

A  comparison  of  trends  in  ACAT  II  and  MDAP  performance  is  useful,  but  a  more 
direct  comparison  of  MDAP  and  ACAT  II  performance  on  cost  growth  and  quantity 
can  provide  clearer  insight  into  the  relative  performance  of  the  two  types  of  programs. 
To  provide  this  direct  comparison,  we  evaluate  a  selected  set  of  MDAPs  using  an 
approach  identical  to  that  used  to  evaluate  ACAT  II  programs  in  the  previous  chapter. 
We  intentionally  do  not  rely  on  SARs  or  DAMIR — data  sources  that  are  traditionally 
used  for  research  on  MDAP  performance  but  are  not  available  for  ACAT  II  programs. 
Programs  are  selected  across  military  departments  and  procurement  program  catego¬ 
ries  using  the  budget  Exhibit  P-1.  A  range  of  different  program  types  is  included  to 
increase  the  generalizability  of  results.  We  selected  both  large  and  smaller  MDAPs, 
but  we  intentionally  exclude  programs  with  serious  and  well-publicized  breaches,  since 
they  may  be  outliers,  and  their  inclusion  could  result  in  a  biased  assessment  of  MDAP 
programs  overall.  We  rely,  as  in  the  previous  chapter,  on  the  budget  Exhibit  P-1  and 
Exhibit  P-40  to  collect  annual  cost  and  procurement  quantity  data  and  to  build  time 
series  datasets  that  track  unit  cost  growth  over  the  life  of  each  program  in  the  set. 
Finally,  also  to  ensure  comparability  of  the  two  selected  sets  of  equipment,  we  do  not 
include  RDT&E  costs  and  assess  programs  only  after  they  have  entered  production. 

We  selected  MDAP  programs  to  include  from  across  military  departments  and 
procurement  program  categories  but  exclude  the  most  well-known  and  problematic 
MDAPs.  We  selected  a  total  of  seven  MDAPs:  the  light  utility  helicopter  (LUH),  joint 
direct  attack  munition  (JDAM),  High-Mobility  Artillery  Rocket  System  (HIMARS), 
the  Virginia-class  submarine,  the  MQ-9,  the  C-27J,  and  the  standard  missile  (SM-2 
and  SM-6).  As  in  the  case  of  the  ACAT  II  set,  we  selected  programs  from  a  mix  of 
military  departments  and  commodity  types.  These  programs  are  classified  and  cat¬ 
egorized  in  Table  3.5.  In  this  section,  we  assess  these  MDAP  programs,  using  unit 
cost  graphs  similar  to  those  presented  for  ACAT  IIs.  Once  again,  the  assessments  and 
observations  that  we  make  about  performance  are  specific  to  our  set  of  programs,  but 
the  method  we  used  to  select  MDAPs  from  the  set  of  programs  (selecting  from  each 
department,  across  procurement  types  and  levels  of  complexity)  should  allow  for  some 
generalizability.  In  our  evaluation  of  MDAPS,  we  do  not  include  program  narratives 
but  instead  focus  more  narrowly  on  program  performance,  including  cost  growth  and 
annual  quantity  procured.  We  chose  not  to  include  this  additional  detail  because  infor¬ 
mation  on  MDAP  program  history  is  readily  and  publicly  available. 

Comparing  Technological  Complexity 

One  concern  about  the  comparison  of  MDAPs  and  ACAT  IIs  is  that  MDAPs  are  fun¬ 
damentally  more  complex  than  ACAT  II  programs  and  so  more  likely  to  experience 
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Table  3.5 

MDAP  Program  Selections 


Program 

Service 

Procurement  Type 

C-27J 

Air  Force 

Aircraft 

JDAM 

Air  Force 

Missiles 

MQ-9 

Air  Force 

Aircraft 

LUH 

Army 

Aircraft 

HIMARS 

Army 

Weapons  and  tracked 
combat  vehicles 

Standard  missile 
(SM-2,  SM-6) 

Navy 

Weapons 

Virginia- class  submarine 

Navy 

Shipbuilding  and  construction 

cost  overruns  and  slippage  during  development,  production,  and  modernizations.  To 
assess  and  control  for  this,  we  collected  information  on  the  technical  specifications  of 
our  set  of  MDAP  and  ACAT  II  programs  from  Jane’s  defense  database  and  then  across 
the  two  sets  of  programs.  Table  3.6  shows  that,  although  the  technical  characteristics 
differ  in  each  sample,  the  two  sets  share  a  number  of  similar  technical  features  and 
overall  an  apparently  similar  level  of  complexity.  This  suggests  that  the  technological 
demands  of  our  two  sets  of  equipment  are  largely  similar  and  should  not  affect  our 
results. 

Overview  and  Summary  of  Results 

Table  3.7  summarizes  our  assessment  of  the  MDAP  programs  that  we  included  in  our 
analysis.  The  programs  are  again  distributed  across  the  three  performance  categories, 
with  two  showing  no  serious  concerns,  four  showing  minor  concerns,  and  three  show¬ 
ing  some  more  significant  concerns.  Overall,  the  MDAPs  in  the  selected  perform  rea¬ 
sonably  well.  Although  a  few  show  some  cost  growth  over  their  program  life  (including 
projected  out-years)  and  some  have  experienced  periods  of  instability,  several  also  show 
a  decrease  in  average  unit  cost.  However,  as  will  be  clear  from  the  program  assessments 
presented  in  this  section,  they  also  do  not  appear  to  perform  significantly  better  than 
the  ACAT  II  set  of  programs,  despite  being  subject  to  more  formal  reviews  and  more 
rigorous  reporting  and  control  requirements. 

MDAP  Program  Assessments 
No  Serious  Concerns 

Virginia- Class  Submarine,  Navy,  Shipbuilding  and  Conversion.  There  are  no  concerns 
associated  with  the  Virginia-class  submarine  program  (Figure  3.17),  which  shows 
stable  unit  cost  and  quantity  procured  since  about  2001.  The  program  experienced  an 
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Table  3.6 

Comparative  Technological  Complexity 


Technical  Characteristics  of 

Technical  Characteristics  of 

Selected  ACAT  II  Programs 

Selected  MDAPs 

Aircraft 

Moveable  delta  wings,  truncated  delta  fins 

Self-loading  guns 

Infrared  (1 R)  seeker  and  camera 

Main  rotor  hub  and  blades 

Charge-coupled  device  (CCD)  color  TV  with  two- 
CCD  switcher 

IR  camera 

RS-232  interface 

Precision  strike  capabilities 

Compact  guidance  and  control  unit 

Self-propelled  rocket  launcher 

Missiles  and  Weapons 


Guidance  system  with  course  alterations 

Single,  two  color  imaging  infrared  (IIR)-seeker 
with  midcourse  correction  or  command,  inertial, 

Radio  frequency  (RF)  seeker 

and  global  positioning  system  (GPS) 

High  Explosive  Anti-Tank  warhead 

Strakes  to  improve  life  and  range 

Four-channel,  switchable  RF  command  link  and 
GPS-based  navigation 

GPS-aided  guidance  electronics 

Proximity  sensor  detonation  system 

Ejectable  cover,  kinetic 

Tracked  Combat  Vehicles 


Rapid  ordnance  removal  system 

Upgraded  suspension  and  fishplates  to  stabilize 
vehicles  during  weapon  firing 

Remote  control  and  heat  sensors 

Modular  construction 

Direct  view  optical  scope 

V-shaped  hull 

Quick-change  machine  gun  barrel 

Remotely  controlled  weapons  stations 

Explosive  reactive  armor 

Energy  absorbing  seats 

Linear  demolition  charge  system 

Advanced  protective  armor 

Full  width,  surface  mine  ploughs 

On-board  vehicle  power 

Shipbuilding  and  Conversion 


Double  rudders  and  planning  stern  seal 

Modular  construction  design 

Reduced  noise  gearbox 

Anechoic  coating 

Four-blade,  variable  reversible-pitch  propellers 

Lock-in/lock-out  hull 

Double-inlet  centrifugal  fan 

Large  diameter  launch  tubes 

Command  and  control  systems  module 

Universal  modular  mast 
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Table  3.7 

MDAP  Program  Assessments 


No  Serious  Concerns 

Minor  Concerns 

Some  Concerns 

Virginia- class  submarine 

JDAM 

MQ-9 

HIMARS 

Standard  missile  (SM-2  and  SM-6) 

C-27J 

LUH 

Figure  3.17 

Unit  Cost  and  Quantity,  Virginia-C\ass  Submarine 
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overall  decrease  in  unit  cost  since  its  first  year  and  is  projected  to  remain  stable  in  the 
out  years. 

HIMARS,  Army,  Weapons  and  Tracked  Vehicles  Procurement.  Figure  3.18  sug¬ 
gests  no  serious  concerns  about  the  HIMARS  program.  The  graph  suggests  that  unit 
cost  is  stable  and  even  declines,  over  the  life  of  the  program,  and  quantity  procured 
rises  until  2009  before  declining  slightly  through  the  final  year  of  available  data,  2011. 
Although  there  appears  to  be  a  slight  increase  in  unit  cost  associated  with  this  drop  in 
quantity  procured,  it  is  relatively  small.  Furthermore,  there  is  no  acquisition  planned 
after  2011,  so  the  effects  of  any  cost  increase  appear  truncated. 

C27J,  Air  Force,  Aircraft  Procurement.  Although  an  assessment  of  the  C-27J  pro¬ 
gram  is  constrained  by  the  very  limited  amount  of  data  available,  we  believe  there  is  no 
cause  for  concern  with  the  program.  Figure  3.19  shows  that  hardware  unit  costs  remain 
consistent  over  the  three  years  of  procurement,  although  the  number  of  aircraft  pro- 
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Figure  3.18 

Unit  Cost  and  Quantity,  HIMARS 


Figure  3.19 

C-27J  Unit  Cost  and  Quantity 
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cured  each  year  increases.  The  increased  FY2012  gross  unit  cost  is  due  to  an  additional 
$18  million  in  engineering  change  orders,  $18  million  in  organic  depot  activation,  $82 
million  in  aircrew  training  devices,  and  changes  to  interim  contractor  support,  rather 
than  to  production  problems  or  inefficiencies.  None  of  these  expenses  are  present  in  the 
first  two  years  of  procurement.  If  costs  such  as  these  continue  to  push  overall  program 
costs  up  in  future  years,  this  may  be  some  cause  for  addition  investigation.  As  of  now, 
however,  the  program  appears  to  be  performing  reasonably  well. 

Some  Concerns 

JDAM,  Air  Force,  Missile  Procurement.  Figure  3.20  suggests  some  cause  for  minor  con¬ 
cerns  associated  with  the  performance  of  the  JDAM  program,  including  an  increase 
in  unit  cost  starting  in  2011  and  continuing  into  the  future.  Initially,  this  increase 
in  unit  cost  is  associated  with  a  drop  in  quantity  procured.  However,  even  after  pro¬ 
jected  quantity  procured  stabilizes  after  2014,  projected  unit  cost  continues  to  increase. 
Because  this  increase  is  only  projected,  it  is  only  a  minor  concern  at  this  point,  but  one 
that  warrants  continuing  attention.  There  is  also  some  instability  in  annual  quantity 
procured  over  the  life  of  the  program,  including  a  sharp  drop  in  2010-2011.  This  drop 
in  quantity  procured  continues  in  the  projected  years  and  may  reflect  reduced  demand 
after  the  end  of  wars  in  Afghanistan  and  Iraq. 

Standard  Missile,  Navy,  Missile  Procurement.  There  are  at  least  minor  concerns 
associated  with  the  standard  missile  program’s  two  primary  variants,  the  SM-2  and 
SM-6.  Figure  3.21  shows  trends  in  unit  cost  and  quantity  for  the  program  as  a  whole 
aggregating  across  the  different  types  of  standard  missiles.  Although  the  program 

Figure  3.20 

Unit  Cost  and  Quantity,  JDAM 
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Figure  3.21 

Unit  Cost  and  Quantity,  Standard  Missile 
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shows  stable  unit  cost  over  the  period  from  1998  through  2007,  in  2008  it  experi¬ 
ences  a  sharp  increase  in  unit  cost,  because  of  the  transition  from  SM-2  to  the  more 
expensive  SM-6.  An  additional  increase  occurs  in  2010,  and  unit  cost  does  not  return 
to  its  original  level,  despite  a  sharp  increase  in  quantity  procured  starting  in  2010  that 
continues  into  future  years.  However,  unit  cost  does  stabilize  in  the  projected  years. 
This  stabilization,  combined  with  the  fact  that  the  program  has  been  stable  in  the  past, 
moderates  concerns  about  program  performance. 

Assessing  and  explaining  the  overall  program  performance  of  the  standard  mis¬ 
sile  become  much  easier  when  we  inspect  the  two  variants  of  the  standard  missile,  the 
SM-2  and  SM-6,  shown  in  Figures  3.22  and  3.23.  The  SM-2  shows  significant  varia¬ 
tion  in  unit  cost  over  time  as  well  as  both  cost  growth  and  a  decline  in  annual  quan¬ 
tity  procured  starting  in  2009.  The  SM-6  has  many  fewer  years  of  associated  data  but 
similarly  shows  some  variation  and  instability  in  both  unit  cost  and  quantity  procured, 
including  a  sudden  cessation  of  all  procurement  in  2013  (there  is  no  procurement  pro¬ 
jected  in  the  out-years).  However,  the  limited  data  available  prevent  a  decisive  assess¬ 
ment  of  the  SM-6  program.  The  instability  in  cost  and  quantity  observed  in  the  two 
programs  together  are  consistent  with  our  rating  of  minor  concerns  for  the  standard 
missile  program. 

Light  Utility  Helicopter,  Army,  Aircraft  Procurement.  The  LUH  (Figure  3.24) 
shows  little  evidence  of  growth  in  unit  cost  or  problematic  instability  to  suggest  poor 
program  health.  Unit  cost  is  relatively  stable  over  the  life  span  of  the  program,  at  least 
until  the  projected  future  years,  when  quantity  procured  begins  to  fall.  In  these  years, 
most  of  the  money  spent  on  the  program  is  for  peripherals  and  repairs  rather  than  for 
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Figure  3.22 

Unit  Cost  and  Quantity,  SM-2 
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Figure  3.23 

Unit  Cost  and  Quantity,  SM-6 
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new  hardware,  so  the  growth  in  gross  unit  cost  reflects  a  shift  in  what  is  being  procured 
rather  than  any  sort  of  problem  in  program  performance.  Quantity  procured  varies 
more  significantly  over  the  life  of  the  program,  rising  consistently  to  2010  and  then 
falling  afterward  into  the  out-years.  The  projected  cost  growth  in  future  years  creates 
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Figure  3.24 

Unit  Cost  and  Quantity,  Light  Utility  Helicopter 
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some  cause  for  concern,  but  since  this  cost  growth  is  projected  and  has  not  yet  materi¬ 
alized,  these  concerns  are  only  minor. 

Some  Concerns 

MQ-9,  Air  Force,  Aircraft  Procurement.  There  are  some  more  significant  concerns  about 
the  performance  of  the  MQ-9.  Figure  3.25  shows  that  unit  cost  over  the  life  of  the  pro¬ 
gram  fluctuates  rather  substantially  and  shows  a  fairly  consistent  increase  in  the  period 
after  2011,  despite  no  change  in  quantity  procured  over  those  same  years.  Annualized 
quantity  procured  rises  in  2012  but  then  falls  in  2013  estimates  and  remains  at  this 
lower  level  in  future  years.  The  instability  and  projected  increase  in  unit  cost,  together 
with  a  fairly  high  projected  annual  procurement  in  future  years,  create  some  cause  for 
concern  about  the  health  of  the  program. 

One  potential  root  cause  of  cost  growth  and  schedule  delays  for  MQ-9  is  its 
rapid  fielding.  The  program  began  in  January  2002  with  system  development  starting 
in  February  2004.  Because  of  urgent  operational  need  and  direction  from  the  com¬ 
mander  of  Air  Combat  Command,  the  MQ-9  was  fielded  with  incomplete  testing, 
before  all  technologies  were  mature  and  even  before  the  completion  of  design  draw¬ 
ings.57  Unless  the  system  is  given  additional  development  time,  extensive  testing,  and 
refined  production  processes,  MQ-9  is  apt  to  encounter  additional  production  delays 


57  Selected  Acquisition  Report,  MQ-9  UAS  Reaper,  December  31,  2011. 
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Figure  3.25 

Unit  Cost  and  Quantity,  MQ-9 


NOTE:  No  gross  cost  data  exist  before  FY2008.  MQ-9  was  procured  under  the  MQ-1  Predator 
appropriation  before  FY2008. 
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and  cost  growth.  These  risks  must  be  balanced  against  the  pressing  need  to  held  the 
system. 

Another  possible  root  cause  for  increasing  unit  costs,  especially  in  later  years  of 
MQ-9’s  life,  are  new,  more  capable,  and  more  expensive  models  of  the  aircraft,  also 
known  as  blocks.  The  original  production  standard  was  known  as  Increment  1  Block  1. 
MQ-9  was  scheduled  to  transition  to  the  more  capable  Increment  1  Block  5  in  FY2012,58 
which  will  Fulfill  all  capabilities  described  in  the  Capability  Production  Document  as 
Reaper  was  originally  planned,  but  these  are  also  more  expensive  because  of  their  more 
advanced  technology.  There  are  plans  to  eventually  move  to  an  Increment  II  Block  10 
with  enhanced  sensors  and  weapons.  As  capabilities  are  incrementally  improved  and 
new  production  blocks  rolled  out,  unit  cost  will  likely  grow. 

Summary 

Overall,  the  MDAPs  in  our  selection  perform  reasonably  well  over  the  time  period 
we  consider,  specifically,  once  they  have  entered  into  production  (we  did  not  consider 
the  development  phase  or  any  problems  these  programs  encountered  at  that  time). 
Although  several  programs  had  some  areas  of  concern,  including  some  evidence  of  cost 
growth  over  time  and  instability  in  procurement  quantity,  the  problems  did  not  yet 
appear  to  be  severe  or  even  close  to  Nunn-McCurdy  breach  thresholds. 


Department  of  the  Air  Force,  FY2012  Budget  Estimates  Justification  Book,  Aircraft  Procurement,  February 
2011. 
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Several  other  observations  emerge  from  our  assessment  of  MDAP  programs. 
First,  returning  to  the  question  of  how  program  size  is  related  to  cost  growth,  we  found 
some  evidence  that  smaller  MDAPs  may  be  more  prone  to  unit  cost  growth  over  their 
program  life,  a  finding  consistent  with  previous  work  on  MDAPs  and  different  from 
the  pattern  observed  in  our  set  of  ACAT  IIs.  For  most  programs,  average  unit  cost  and 
unit  cost  in  the  final  year  of  the  program  were  lower  than  unit  cost  in  year  one  (in  con¬ 
stant  dollar  terms).  Programs  that  did  experience  cost  growth  included  both  programs 
with  significant  total  costs,  such  as  the  C-27J,  and  those  with  smaller  total  program 
costs,  such  as  the  standard  missile.59  Since  the  cost  increase  in  the  C-27J  program  was 
due  to  service  contracts,  not  hardware,  the  balance  tips  slightly  toward  a  conclusion 
that  smaller  MDAPs  experience  greater  cost  growth.  The  same  caveats  noted  above 
are  important  when  comparing  these  findings  to  previous  work  on  the  relationship 
between  size  and  MDAP  performance.  First,  we  did  not  include  RDT&E  costs  and 
considered  programs  only  once  they  had  entered  into  production.  Second,  as  noted 
above,  the  time  period  considered  in  some  previous  work,  specifically,  that  relating 
cost  growth  and  program  size,  looks  at  programs  from  the  1990s,  when  the  fiscal  and 
competitive  environment  and  the  acquisition  workforce  looked  significantly  different. 

Performance  of  ACAT  II  Programs  Compared  to  MDAPs 

A  comparison  of  the  MDAP  and  ACAT  II  set  of  programs  suggests  that  ACAT  II 
programs  perform  about  as  well  as  MDAPs  overall,  when  both  types  of  programs  are 
assessed  after  they  have  entered  production.  In  both  sets,  once  they  have  entered  into 
production,  programs  perform  reasonably  effectively,  with  some  instability  in  unit  cost 
and  annual  quantity  procured  but  few  instances  of  rapid,  long-term  cost  growth  or 
significant  slippage  in  schedule.  In  both  cases,  cost  growth  and  instability  in  quantity 
procured  are  associated  with  modernizations,  contingency  operations,  and  accelerated 
time  lines  but  tend  to  be  limited  in  size  and  often  duration.  Table  3.8  offers  a  more 
quantitative  way  to  compare  the  performance  of  the  two  programs,  namely,  by  com¬ 
paring  the  aggregate  changes  in  quantity  procured  and  unit  cost  across  the  two  sets 
between  2010  and  2011.  The  table  shows  that  MDAPs  and  ACAT  IIs  trend  in  the 
same  direction  on  these  dimensions.  Quantity  procured  increases,  and  this  increase 
is  significantly  larger  for  the  MDAP  selections.  Unit  cost  for  hardware  rises  in  both 
cases,  and  this  increase  is  also  larger  for  the  MDAP  set.  However,  gross  unit  cost  falls 
for  ACAT  IIs  and  MDAPs.  These  numbers  are  only  rough  estimates  and  are  limited 
in  their  value  by  the  fact  that  they  average  across  a  large  number  of  very  different  pro¬ 
gram  types.  However,  these  estimates  are  sufficient  to  support  our  overall  argument 
and  assessment  that  the  ACAT  II  and  MDAP  sets  perform  similarly  on  a  large  number 
of  important  dimensions. 


59 


The  C-27J  cost  increase  was,  as  noted  above,  largely  due  to  factors  not  associated  with  hardware  or  system 


costs. 
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Table  3.8 

Comparison  of  MDAP  and  ACAT  II  Performance  (FY2010-FY2011) 


Selected  ACAT  II 
Programs 

Selected  MDAP 
Programs 

Quantity 

t  23.0% 

t  77.7% 

Unit  cost  (gross) 

|  4.4% 

|  8.2% 

Total  cost  (gross) 

t  8.8% 

t  50.0% 

As  stated  at  the  start,  adding  new  guidance  and  reporting  and  control  require¬ 
ments  for  ACAT  II  programs  makes  sense  only  if  there  is  evidence  of  a  problem  in  their 
current  performance  that  suggests  that  the  military  departments  are  unable  to  manage 
these  programs  on  their  own  or  that  additional  centralized  oversight  would  solve  the 
performance  problems  faced  by  ACAT  IIs. 


Conclusion 

Our  assessment  suggests  that  ACAT  IIs  reach  a  level  of  performance  currently,  without 
rigorous  centralized  oversight,  that  is  at  least  equal  to  that  of  MDAPs  operating  with 
centralized  reporting  and  control  requirements  and  oversight.  As  a  result,  it  appears 
that  the  military  departments  are  already  doing  a  fairly  effective  job  at  managing  these 
programs  and  that  additional  guidance  may  not  be  warranted.  Furthermore,  the  prob¬ 
lems  that  ACAT  II  programs  do  seem  to  suffer  from  are  diverse  and  program-specific 
and  unlikely  to  be  solved  simply  by  the  addition  of  a  more  centralized  performance 
reporting  system. 


CHAPTER  FOUR 

Framing  Assumptions 


Introduction 

Most  defense  acquisition  programs  face  the  challenge  of  forecasting  cost,  schedule,  and 
technical  expectations  far  in  advance  of  actual  work — sometimes  decades  beforehand. 
Moreover,  programs  must  also  account  for  differences  in  acquisition  strategy  and  mar¬ 
ketplace  environment  in  these  forecasts.  As  a  result,  program  managers  must  make 
assumptions  regarding  these  uncertain  elements  of  program  execution.  Although  nec¬ 
essary,  such  assumptions  expose  the  program  to  risk  of  cost  growth  and  schedule  slip. 

PARCA  office  research  has  noted  that  incorrect  assumptions  (or  conditions  that 
make  the  assumptions  invalid)  can  cause  a  program  to  miss  its  budget  forecasts.  More 
important,  these  incorrect  assumptions  often  lead  to  breaches  of  Nunn-McCurdy 
thresholds  if  they  are  central  to  program  expectations.  Such  key  assumptions  are  called 
“framing  assumptions,”  because  they  so  shape  the  program  performance  expectations 
that,  if  incorrect,  they  lead  to  significant  cost  growth,  performance  shortfalls,  schedule 
slippage,  or  all  three. 

This  exploratory  research  conducted  for  the  PARCA  office  attempts  to  accom¬ 
plish  the  following  goals: 

•  define  framing  assumptions  so  that  others  can  better  identify  them  and  that  there 
are  some  standards  for  consistent  application 

•  explore  the  possibility  of  identifying  framing  assumptions  for  a  sampling  of  pro¬ 
grams  (both  established  and  relatively  new)  using  readily  available  information 
(e.g.,  publicly  available  program  documents,  congressional  testimony,  SARs,  and 
press  articles). 

The  potential  benefit  of  this  research  is  showing  that  the  concept  of  framing 
assumptions  is  a  feasible  and  practical  paradigm  for  understanding  potential  risks  to 
programs.  If  program  risks  can  be  identified  early,  they  can  be  tracked  and  managed 
during  program  execution. 
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Assumptions  in  Planning 

The  defense  acquisition  and  business  literature  touches  indirectly  on  identifying 
assumptions  underlying  planning  and  execution  through  discussions  of  risk  manage¬ 
ment  and  uncertainty  management.1  However,  there  is  little  discussion  on  how  to 
identify,  prioritize,  and  then  vet  assumptions  vital  to  a  project  plan.  Work  by  James 
Dewar  concerning  assumption-based  planning  offers  several  techniques  to  identify 
explicit  or  implicit  assumptions  that  are  both  vulnerable  as  well  as  “load  bearing.”2  In 
APB,  a  planning  assumption  is  defined  as  “a  judgment  or  evaluation  about  some  char¬ 
acteristic  of  the  future  that  underlies  the  plans  of  an  organization.”3 

A  planning  assumption  can  have  several  different  characteristics.  The  first  is 
whether  it  is  explicit  and  articulated  or  implicit.  Particularly  problematic,  implicit 
assumptions  may  be  so  because  of  suppression  for  organizational  or  external  politi¬ 
cal  reasons,  repression  by  planners,  “cultural  blinders,”  or  cognitive  blocks.4  However, 
true  risk  lies  in  whether  an  assumption  is  load  bearing  and  vulnerable.  An  assumption 
is  load  bearing  “if  its  failure  would  require  significant  changes  in  the  organization’s 
plans.”5  Vulnerability  is  inherent  when  plausible  events  could  cause  it  to  fail  within  the 
expected  lifetime  of  the  plan.  According  to  Dewar,  assumptions  identified  as  both  load 
bearing  and  vulnerable  must  receive  special  scrutiny  as  the  project  progresses. 

Dewar  identifies  two  major  sources  for  assumption  identification:  project  docu¬ 
mentation  and  people  involved  in  the  planning.  Drawing  on  these  sources,  it  is  possible 
to  apply  several  techniques  that  vary  as  to  the  time  and  number  of  people  required, 
ease  in  application,  breadth  of  applicability,  and  point  in  time  during  the  project  when 
assumption  identification  should  be  used.6  The  techniques  Dewar  lists  include  the  fol¬ 
lowing:  Telling  actions  the  long  way,  looking  for  wills/musts,  rationalizing  a  plan, 
asking  journalist’s  questions,  strategic  assumption  surfacing  and  testing,  driving  force 
analysis,  core  belief  identification  squad,  and  an  annual  key  bets  contract.  Particularly 
when  using  documentation,  the  identification  of  wills/musts  and  their  derivatives  pres¬ 
ent  an  easy  method  to  identify  explicit  assumptions.  “Rationalizing  a  plan”  entails  list- 


1  See,  for  example,  Young  Hoon  Kwak  and  Brian  M.  Smith,  “Managing  Risks  in  Mega  Defense  Acquisition 
Projects:  Performance,  Policy,  and  Opportunities,”  International  Journal  of  Project  Management,  Vol.  27,  2009, 
pp.  819-820;  Chris  Chapman,  “Key  Points  of  Contention  in  Framing  Assumptions  for  Risk  and  Uncertainty 
Management;  “  International  Journal  of  Project  Management,  Vol.  24,  2006,  pp.  303-313;  Corporate  Strategy 
Board,  “Proceeding  in  Daylight:  Frontier  Practices  for  Challenging  Strategic  Assumptions,”  Washington,  D.C.: 
Foundations  of  Strategy  Development  series,  1999. 

2  James  A.  Dewar,  Assumption-Based.  Planning:  A  Tool  for  Reducing  Avoidable  Surprises,  New  York:  Cambridge 
University  Press,  2002. 

3  Dewar  2002,  p.  14. 

4  Dewar  2002,  pp.  20-21. 

5  Dewar  2002,  p.  22. 

6  Dewar  2002,  p.  57 
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ing  separately  the  actions  the  organization  will  take  and  assumptions  about  the  future 
and  then  connecting  the  assumptions  to  the  actions.  Some  assumptions  will  connect 
strongly,  weakly,  or  not  at  all,  and  some  actions  may  not  have  any  connected  assump¬ 
tions.  These  isolated  actions  represent  instances  of  potential  implicit  assumptions  and 
should  be  explored  further. 

Once  the  assumptions  are  identified,  Dewar  suggests  identifying  “signposts,”  or 
indicators,  or  warning  signals  that  the  validity  or  vulnerability  of  an  assumption  is 
changing.  To  create  signposts,  analysts  can  use  formal  indicator  systems,  such  as  eco¬ 
nomic  indices;  derive  them  from  formal  project  goals,  targets,  or  objectives;  or  simply 
brainstorming  paths  or  turning  points  where  assumptions  could  fail.  The  identified 
signposts  allow  for  “reality  checks”  as  the  project  and  its  environment  evolve. 


Definition  of  a  Framing  Assumption 

Before  we  can  explore  example  framing  assumptions,  we  first  need  to  define  what  a 
framing  assumption  is.  Such  a  definition  is  particularly  challenging  in  that  it  is  easy  to 
recognize  a  key  assumption  (particularly  after  the  fact  when  it  is  shown  to  be  invalid) 
but  not  to  come  up  with  an  all-encompassing  definition.  In  fact,  the  definition  for  a 
framing  assumption  was  the  last  task  we  undertook  after  having  looked  at  a  range  of 
assumptions  on  programs.  We  arrived  at  the  following  definition: 

A  framing  assumption  is  any  explicit  or  implicit  assumption  that  is  central  in  shaping 
cost,  schedule,  or  performance  expectations.  In  other  words,  framing  assumptions  can  be 
viewed  as  “bets”  that,  if  not  realized,  can  lead  to  major  problems  in  execution.  Framing 
assumptions  may  change  over  the  course  of  execution  or  new  ones  can  be  added.  A  few 
criteria  further  refine  this  definition: 

•  Determinative:  The  consequences  of  the  assumption  being  wrong  will  signifi¬ 
cantly  affect  the  program  in  a  way  that  matters  (e.g.,  significant  cost  growth  or 
schedule  slippage).  In  other  words,  the  assumption  is  important  to  the  success 
of  the  program.  For  example,  an  incorrect  framing  assumption  could  lead  to  a 
significant  or  critical  Nunn-McCurdy  breach.  The  assumption  must  be  explicitly 
tied  to  how  it  affects  the  value  chain  (e.g.,  a  particular  acquisition  strategy  will 
result  in  lower  procurement  cost). 

•  Unmitigable:  The  consequences  are  unavoidable  if  the  assumption  is  wrong 
through  other  actions.  There  is  no  easy  workaround  or  mitigating  action  the  PM 
can  take.  For  example,  the  electromagnetic  launch  system  not  working  on  the 
carrier  CVN  78  would  have  unavoidable  cost,  schedule,  and  technical  implica¬ 
tions  for  the  carrier  program.  There  is  no  simple  workaround — going  back  to  a 
steam  catapult  option  would  be  very  costly  and  time-consuming. 

•  Uncertain:  The  outcome  or  certainty  with  respect  to  the  assumption  is  unknown. 
There  is  some  inherent  risk.  For  example,  a  framing  assumption  is  not  a  funda- 
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mental  physical  law  or  property,  such  as  the  gravitational  constant.  Indeed,  if  this 
constant  were  to  change,  there  would  be  consequences;  but  it  will  not  happen. 

•  Independent:  The  consequences  of  the  assumption  failing  or  being  realized  do 
not  hinge  on  other  events  or  a  chain  of  events.  As  a  simple  example,  a  framing 
assumption  is  not  “the  contract  price  is  stable.”  Price  stability  could  depend  on 
many  separate  issues,  such  as  industrial  base  stability,  inflation  of  materials,  or 
technical  maturity. 

•  Distinctive:  A  framing  assumption  should  generally  set  a  program  apart  from 
other  programs  (or  groups  of  programs).  So  a  framing  assumption  is  not  “the 
contractor  will  perform  well,”  because  that  assumption  is  common  to  all  pro¬ 
grams.  But  a  framing  assumption  might  be,  “the  contractor  is  capable  of  success¬ 
fully  producing  a  type  of  system  that  it  has  not  produced  in  the  past.”  Think  of 
the  advanced  SEAL  delivery  system  program — the  original  contractor  had  never 
made  a  deep  submergence  vehicle  or  submarine.7 

An  example  of  a  framing  assumption  might  be,  “competitive  prototyping  will 
result  in  a  savings  of  5  percent  in  procurement  cost,”  rather  than,  “the  contractor  will 
perform  well”  (common  to  all  programs).  Another  example  is  that  a  particular  weapon 
system  being  replaced  “will  last  until  2015”  and  all  schedule  plans  are  predicated  on 
that  end  date.  A  framing  assumption  is  not  a  program  characteristic,  e.g.,  the  program 
is  part  of  a  family  of  systems.  As  a  final  example,  a  framing  assumption  might  be  that 
the  program  “can  leverage  specific  COTS  hardware  and/or  software  to  reduce  risk  and 
save  cost.”  A  framing  assumption  is  not  “the  program  is  affordable.” 

To  help  make  the  concept  of  framing  assumptions  more  concrete,  we  organized 
them  into  three  categories,  as  shown  in  Table  4.1.  The  categories  are  technical  (both 
component  and  integration  issues);  management  approaches,  incentives,  and  program 
structures;  and  mission  requirements.  Note  that  the  table  does  not  list  specific  assump¬ 
tions  but  rather  identifies  areas  where  a  program  may  have  framing  assumptions. 
Also,  some  issues  may  not  be  entirely  separable  into  specific  areas  and  drive  multiple 
consequences.  For  example,  quantity  expectations  might  also  affect  industrial  base 
expectations. 

One  important  issue  to  note  is  that  a  framing  assumption  is  not  necessarily  a 
“bad”  thing  or  incorrect.  All  programs  make  assumptions,  and  many  may  turn  out  to 
be  valid.  The  important  point  is  that  program  outcomes  depend  heavily  on  making  the 
correct  framing  assumptions. 

It  is  also  possible  to  go  overboard  in  identifying  framing  assumptions.  The  con¬ 
cept  is  to  develop  a  few  concrete,  high-level  assumptions  that  can  be  easily  tracked.  On 
the  order  of  three  to  five  assumptions  is  probably  a  good  number  for  most  programs. 


7  U.S.  Government  Accountability  Office,  Success  of  Advanced  SEAL  Delivery  System  Hinges  on  Establishing  a 
Sound  Contracting  Strategy  and  Performance  Criteria,  Washington,  D.C.,  GAO-07-745,  2007b. 
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Table  4.1 

Example  Program  Areas  for  Framing  Assumptions 


Technological 

Management,  Incentives, 
and  Program  Structures 

Mission  Requirements 

Manufacturing  expectations 

Program/interprogram 

Stability  and  compatibility  of 

Testing  expectations 

dependencies 

operational  needs 

Contractual  incentives  and 

Quantity 

Industrial  base  and  market 

strategies 

expectations 

Organizational  management 

Capabilities 

Technical  approach 

structures 

Joint/international  needs 

Risk  expectations 

Risk  assessment 

Possibility  of  substitute  item/system 

Simulation  utility 

Risk  management 

Interface  and  environment 
management 

Example  Programs 

Approach 

The  PARCA  office  identified  five  exemplar  programs  for  which  it  wanted  us  to  explore 
developing  framing  assumptions.  These  programs  are  as  follows: 

•  advanced  pilot  training  (APT)  system 

•  joint  light  tactical  vehicle  (JLTY) 

•  joint  precision  approach  and  landing  system  (JPALS) 

•  LCS  modules 

•  space  fence. 

Part  of  the  ground  rules  for  this  analysis  was  to  develop  these  framing  assump¬ 
tions  independently  from  program  offices  or  any  other  organization  in  OSD.  To  do  so, 
we  reviewed  available  program  literature.  We  specifically  used  the  following  sources: 
ADMs,  open  source  program  briefings,  trade  literature  {Inside  Defense,  Jane’s,  or  other 
relevant  trade  literature),  DAES,  SARs,  Acquisition  Strategies,  congressional  autho¬ 
rization  and  appropriations  documents,  enabling  concept  documents,  SEPs,  GAO 
reports,  Congressional  Research  Service  reports,  DAES  risk  summary  and  issue  sum¬ 
mary  charts,  Exhibit  R-2,  RDT&E  budget  item  justification  documents,  TEMPs, 
technology  readiness  are  (TRA),  AoA,  and  independent  cost  estimates  (ICEs)  that 
were  available.  Not  all  material  was  available  for  each  program. 

Beyond  these  five  programs,  we  summarize  the  prior  RAND  PARCA  research  on 
root  causes,  by  identifying  framing  assumptions  that  caused  Nunn-McCurdy  breaches. 
We  did  not  do  such  an  extensive  literature  review  for  the  programs  listed  above. 
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Note  that  the  framing  assumptions  are  our  interpretation  of  these  sources  and 
required  that  we  make  inferences  about  program  expectations.  None  of  the  informa¬ 
tion  we  reviewed  explicitly  listed  key  assumptions,  nor  did  we  interview  program  office 
personnel  to  verify  our  findings.  Thus,  our  assessments  may  not  be  inclusive  or  list  all 
the  important  framing  assumptions.  Also,  note  that  it  is  easier  to  recognize  a  failed  or 
incorrect  assumption  after  the  fact  than  it  is  to  recognize  a  correct  or  valid  one.  There¬ 
fore,  our  analysis  is  more  likely  to  have  identified  incorrect  assumptions. 

Advanced  Pilot  Training  System 

The  APT  is  a  pre-MDAP  Air  Force  aircraft  and  ground  training  system  that  will 
replace  the  T-38  aircraft  training  system.  The  last  awarded  milestone  was  Milestone 
A  on  May  10,  2010. 8  APT  will  be  an  integrated  family  of  systems  that  will  use  both 
ground  (simulators  and  computer-based  training)  and  flight  training  systems.9  The 
program  is  expected  to  supply  an  advanced  trainer,  known  as  the  T-X  aircraft,  for 
the  fighter/bomber  APT  track.  Market  research  was  conducted  in  2008  for  possible 
sources  for  the  APT;  however,  initial  operational  capability  was  moved  from  2017  to 
2020  because  of  budgetary  concerns.10 

Given  the  early  stage  of  program  planning,  little  information  was  available  to  gen¬ 
erate  framing  assumptions  from  open  source  reporting;  however,  we  were  able  to  glean 
some  framing  assumptions  that  are  presented  in  Table  4.2. 

The  first  framing  assumption  we  were  able  to  identify  for  the  APT  is  that  it  will 
be  nondevelopmental,  meaning  that  the  program  will  leverage  commercial  or  foreign 
aircraft’s  airframe  and  avionics  capabilities.* 11  This  is  a  technologically  related  assump¬ 
tion.12  Use  of  COTS  and  other  nondevelopmental  items  is  frequently  done  in  defense 
acquisition  to  maintain  lower  costs.  This  strategy  has  produced  mixed  results.13  In  this 


8  Ashton  B.  Carter,  “Memorandum  for  Secretary  of  the  Air  Force  and  Secretary  of  the  Navy;  Subject:  Advanced 
pilot  Training  (APT)  System  Materiel  Development  Decision  (MDD)  Acquisition  Decision  Memorandum,” 
Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics,  May  10,  2010.  Not  available 
to  the  general  public. 

9  Col.  John  Thompson,  “The  Future  of  Air  Force  Education  and  Training:  an  Advance  Planning  Briefing  to 
Industry,”  Randolph,  Tex.:  Air  Education  and  Training  Command,  May  12,  2009,  p.  12. 

10  Gabe  Starosta,  “Cautions  Against  Living  ‘on  the  Leading  Edge  of  Technology’:  AETC  Chief  Confident  in 
T-38  Capability  with  IOC  Delayed  Until  2020,”  Inside  the  Air  Force,  March  2,  2012. 

11  Carter,  2010. 

12  One  could  alternatively  create  a  different  framing  assumption  under  mission  requirements  that  the  “Mis¬ 
sion  requirements  can  be  met  with  non-developmental  aircraft.”  Sometimes  framing  will  not  cleanly  fit  into  one 
specific  category.  But  this  indeterminacy  is  not  critical;  it  is  more  important  to  identify  the  assumption  than  to 
categorize  it. 

13  See,  for  example,  Office  of  the  Under  Secretary  of  Defense  for  Acquisition  Technology,  and  Logistics,  “Buying 
Commercial:  Gaining  the  Cost/Schedule  Benefits  for  Defense  Systems,”  Report  of  the  Defense  Science  Board  Task 
Force  on  Integrating  Commercial  Systems  into  the  DOD,  Effectively  and  Efficiently,  Washington,  D.C.,  February 
2009. 
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Table  4.2 

APT  Framing  Assumptions 


Technological 

Management,  Incentives, 
and  Program  Structures 

Mission  Requirements 

Training  aircraft  (both 
airframe  and  avionics)  will 
be  nondevelopmental 

Possible  customers  (e.g.,  the 

U.S.  Navy  and  foreign  militaries) 
will  want  to  buy  the  APT  with 
minimal  modifications 

Current  training  scenario  is  valid: 

Use  of  existing  T-38  can  be  extended 
until  2020 

Simulators  can  be  used  instead  of 
actual  flight  time  (to  save  money) 

particular  program,  using  a  proven  airframe  may  produce  a  positive  outcome  if  mini¬ 
mal  modifications  are  needed  to  the  weight  and  size  of  the  airframe.  Using  COTS  for 
the  avionics  will  also  be  positive  if  there  are  minimal  modifications  and  if  integration 
does  not  prove  to  be  difficult. 

The  second  framing  assumption  falls  under  the  “management,  incentives,  and 
program  structures”  category.  This  framing  assumption  for  the  APT  program  assumes 
that  any  possible  future  customers  for  the  APT  (outside  the  Air  Force)  would  want  to 
buy  the  APT  with  minimal  modifications.14  Such  parties  may  include  other  services 
or  foreign  militaries.  This  is  another  cost-savings  assumption  that  reflects  a  frequently 
used  cost  saving  mechanism  in  defense  acquisition.  By  producing  more  units  of  the 
same  aircraft  for  outside  parties,  the  Air  Force  would  be  able  to  maintain  lower  unit 
costs  in  production. 

The  final  framing  assumption  relates  to  the  systems  mission  requirements,  namely, 
that  the  current  training  scenario  is  valid:  (a)  the  use  of  existing  T-38  can  be  extended 
until  2020,  and  (b)  simulators  can  be  used  instead  of  actual  flight  time  (to  save  money). 
In  the  case  of  the  APT,  the  Air  Force  made  a  recent,  critical  assumption  that  the  cur¬ 
rent  platform  that  is  being  used  for  training,  the  T-38,  is  sufficient  to  meet  operational 
needs  until  2020,  when  the  APT  will  reach  initial  operational  capability.  Recently,  the 
Air  Force 

opted  to  delay  the  initial  operational  capability  of  a  replacement  trainer  aircraft 
by  three  years  until  fiscal  year  2020  .  .  .  [because]  the  T-38  trainer’s  airframe  is  in 
good  enough  shape  to  operate  until  the  end  of  the  decade  . . .  AETC’s  commander, 
acknowledged  that  .  .  .  the  delay  will  cause  few  problems  in  terms  of  sustaining 
the  T-38.15 

As  long  as  the  Air  Force’s  calculations  for  the  T-38  are  accurate,  there  should  be 
little  problem  with  this  framing  assumption;  however,  if  the  APT  is  delayed  even  fur¬ 
ther  because  of  budgetary  constraints  or  canceled  outright,  then  the  absence  of  a  read- 


14  Carter,  2010. 

15  Starosta,  2012. 
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ily  available  alternative  may  be  problematic  for  the  Air  Force.  Also  as  part  of  the  mis¬ 
sion  requirements’  risk  is  the  use  of  simulators  instead  of  actual  flight  time.  One  reason 
for  the  use  of  simulators  is  to  save  flight  time  costs.  In  addition,  given  the  changing 
environment  for  technology,  the  Air  Force  is  focusing  on  other  ways  to  use  technology 
through  the  following: 

•  instructional  design 

•  knowledge  systems 

•  virtual  environments 

•  mobile  learning 

•  simulations  and  gaming.16 

APT  is  one  program  that  the  Air  Force  has  developed  in  a  way  that  shifts  its 
acquisition  to  reflect  changing  technologies. 

Joint  Light  Tactical  Vehicle 

The  JLTV  is  a  pre-MDAP,  joint  acquisition  program  under  the  Army  and  the  Marine 
Corps  that  seeks  to  replace  the  11  versions  of  the  high  mobility,  multi-wheeled  vehicle 
that  have  been  in  service  since  1985.  A  ground  combat  vehicle,  the  JLTV  is  planned  to 
provide  substantial  improvements  over  the  existing  light  tactical  wheeled  fleet  in  tech¬ 
nology,  operational  capability,  and  survivability.  In  particular,  the  Army  has  sought  to 
acquire  the  JLTV.  It  was  awarded  Milestone  A  on  December  22,  2007. 

One  major  technological  framing  assumption  entailed  embracing  benefits 
implied  with  the  “incremental/open  architecture”  concept  (see  Table  4.3).  This  con¬ 
cept  has  been  used  in  other  acquisition  programs  as  well,  with  mixed  results.17  In  par¬ 
ticular,  the  JLTV  was  envisioned  to  have  an  initial  “basic  capability”  with  the  poten¬ 
tial  to  add  “enhanced  force  protection,  increased  fuel  efficiency,  greater  payload,  and 
other  improvements”  to  integrate  future  technologies.18  Basically,  capabilities  would 
be  scaled  to  fit  various  operational  needs.  Another  technological  framing  assumption 
is  that  that  competitive  prototyping  will  also  reduce  risk  and  cost.  Multiple  vendors, 
it  was  assumed,  would  be  able  to  “increase  their  knowledge  of  the  needed  technolo¬ 
gies,  determine  the  technologies’  maturity  level,  and  determine  which  combination  of 
requirements  were  achievable.”19  This  assumption  is  also  born  out  of  the  belief  that  the 


16  Thompson,  2009,  p.  9. 

17  For  example,  see  Mark  A  Lorell,  Julia  F.  Lowell,  and  Obaid  Younossi,  Evolutionary  Acquisition:  Implementa¬ 
tion  Challenges  for  Defense  Space  Programs,  Santa  Monica,  Calif.:  RAND  Corporation,  MG-431-AF,  2006. 

18  U.S.  Government  Accountability  Office,  Defense  Acquisitions:  Issues  to  be  Considered  as  DOD  Modernizes  Its 
Fleet  of  Tactical  Wheeled  Vehicles,  Washington,  D.C.,  GAO-11-33,  November  2010b,  p.  11;  LTC  Wolfgang  Peter- 
mann  and  LtCol  Ruben  Garza,  “JLTV  EMD  Industry  Day,”  briefing  presented  on  April  26,  2010. 

19  U.S.  Government  Accountability  Office,  Defense  Acquisitions:  Future  Ground-Based  Vehicles  and  Network  Ini¬ 
tiatives  Face  Development  and  Funding  Challenges,  GAO-12-181T,  Washington,  D.C.,  October  2006,  p.  10. 
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Table  4.3 

JLTV  Framing  Assumptions 


Technological 

Management,  Incentives, 
and  Program  Structures 

Mission 

Requirements 

Incremental/open  architecture  will 
reduce  risk  and  allow  for  more 
efficient  upgrades 

A  joint  Army  and  Marine  Corps 
program  will  save  money  as 
requirements  are  compatible 

The  services  have  effectively 
assessed  long-term  versus 
short-term  needs 

Competitive  prototyping  will  reduce 
risk/cost 

Export  opportunities  could 
decrease  cost 

use  of  “one  bidder  based  on  a  paper  proposal  has  proven  to  be  a  formula  for  risk,  cost 
growth  and  schedule  delays.”20 

The  management  framing  assumption  is  that  a  partnership  between  the  Army  and 
the  Marine  Corps,  despite  differing  requirement  tradeoffs  between  the  two  services, 
would  reduce  overall  program  costs  through  the  use  of  similar  equipment.21  These 
two  framing  assumptions,  that  “unique  service  requirements  have  been  minimized”22 
and  that  competition  within  the  industry  would  overcome  the  technical  difficulties  in 
merging  the  services’  differing  mission  goals,  reinforced  each  other.  In  particular,  force 
protection  and  transportability  represented  the  greatest  challenge  to  joint  development. 
The  Army’s  key  goal  was  “scalable  protection,”23  yet  the  resulting  armor  requirement 
drove  up  the  total  weight  of  the  vehicle  and  ran  directly  counter  to  the  Marine  Corps 
requirement  that  the  vehicle  be  transportable  by  the  CH-47  and  CH-53  helicopters 
as  well  as  the  C-130  cargo  aircraft.  After  ACTD,  size  requirements  were  adjusted 
in  an  effort  to  harmonize  the  tradeoffs  between  the  two  requirements,  although  the 
decreased  size  led  to  limited  space  for  other  mission-essential  equipment  and  payload.24 


20  As  stated  by  Young,  2008,  p.  71.  According  to  the  PEO  for  Land  Systems,  “the  purpose  of  the  TD  phase  was 
to  better  inform  the  requirements  communities  by  allowing  them  to  get  a  glimpse  of  what  the  realm  of  the  pos¬ 
sible  with  respect  to  what  those  requirements  cost.”  “Amphibious  Combat  Vehicle  AOA  Scheduled  to  Wrap  Up 
in  June,”  InsideDefense.com:  Inside  the  Navy,  March  9,  2012. 

21  “It  always  bothers  me  that  we  may  not  be  totally  efficient  in  buying  things  that  should  maybe  be  joint.  In 
other  words,  the  idea  of  the  Army  and  Marines  using  similar  equipment  is  a  good  idea.  But  I  value  the  Marines’ 
understanding  of  their  mission,  which  is  slightly  different  than  the  Army  mission,”  according  to  Army  acquisition 
executive  Malcolm  O’Neill.  Sebastian  Sprenger  and  Tony  Bertuca,  “Army  Acquisition  Chief  Backs  JLTV,  But 
Acknowledges  Unknowns,”  InsideDefense.com:  Inside  the  Army,  October  18,  2010. 

22  JLTV  R-2  Budget,  February  2012,  p.  1. 

23  Department  of  the  Army,  “The  Army  Tactical  Wheeled  Vehicle  (TWV)  Strategy,”  2010,  p.  13. 

24  COL  Steve  Meyers  and  LtCol  Ben  Garza,  “TWV  Transformation  Efforts,”  presentation  at  the  NDIA  TWV 
Conference,  February  6,  2007,  slides  12,  15. 
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Other  management/program  structure  framing  assumptions  included  the  belief 
that  export  opportunities  to  countries  such  as  Australia  could  decrease  cost.25  In  addi¬ 
tion,  the  program  was  viewed  as  a  good  “test-bed”  for  new  OSD  acquisition  initiatives 
to  improve  program  performance.  The  JLTV  was  labeled  specifically  as  the  “designated 
pilot  for  new  OSD  acquisition  initiatives”  such  as  “Concept  Decision,  Fully  Burdened 
Cost  of  Fuel,  Competitive  Prototyping,  (and)  Increased  RAM.”  26 

In  terms  of  mission  requirement  framing  assumptions,  “the  JLTV  CONOPS 
(were)  expected  to  evolve  in  synchronization  with  the  program  development  .  .  .  the 
design,  test,  and  trade  study  efforts  in  the  TD  are  expected  to  influence  the  solidifi¬ 
cation  of  CONOPS.”27  Balancing  longer-term  requirements  with  shorter-term  oper¬ 
ational  needs  was  deemed  feasible,  as  the  heavier  mine  resistant  ambush  protected 
vehicles  “fill(ed)  a  near-term,  urgent  joint  service  requirement”  for  the  high  impro¬ 
vised  explosive  device  threat,  whereas  the  JLTV  was  supposed  to  be  used  in  other 
threat  environments.  The  unexpected  rapid  acquisition  and  deployment  of  the  mine- 
resistant  ambush  protected  vehicle,  however,  altered  requirements  and  expectations  for 
the  JLTV,  particularly  in  its  curb  weight  and  transportability  requirements.28 

Joint  Precision  Approach  and  Landing  System 

The  JPALS  program  is  an  ACAT  ID  MDAP.  The  Navy  is  the  lead  service,  but  both  the 
Air  Force  and  Army  provide  additional  support  to  the  program.  It  can  be  categorized 
as  a  command  and  control  program  but  is  specifically  a  GPS -based  precision  approach 
and  landing  system.29  According  to  the  mission  description  in  the  SAR,  this  system 

will  replace  several  aging  and  obsolete  aircraft  landing  systems  with  a  family  of  sys¬ 
tems  that  is  more  affordable  and  will  function  in  more  operational  environments, 
and  support  all  Department  of  Defense  (DoD)  Land  and  Sea  Based  applications 
.  .  .  [and]  will  eventually  support  unmanned  and  highly  automated  aircraft.  .  .  . 

The  approved  JPALS  Acquisition  Strategy  has  acquisition  broken  into  seven  incre¬ 
ments,  based  on  technology  maturity  and  Service  needs,  increment  i,  Sea  Based 


25  “International  participation  in  the  JLTV  program  will  reduce  overall  program  risk  through  the  testing  and 
evaluation  of  additional  prototype  vehicles,”  according  to  Lt.  Col.  Wolfgang  Petermann,  Army  product  manager 
for  JLTV.  Marjorie  Censer,  “DOD  Inks  Formal  JLTV  Agreement  with  Australia;  More  Partnerships  Planned,” 
InsideDefense.com:  DefenseAlert,  February  26,  2009. 

26  David  Holm,  “‘Milestone  A’  Costing:  Joint  Light  Tactical  Vehicle,  Presentation  for  the  U.S.  Army  Tank- 
Automotive  Command  (TACOM),  slide  6,  n.d. 

27  “JLTV  Systems  Engineering  Plan  Milestone  A,”  November  21,  2007,  p.  25. 

28  “The  Army  and  Marine  Corps  have  agreed  on  a  new  set  of  base  requirements  for  the  Joint  Light  Tactical  Vehi¬ 
cle,  including  a  heavier  curb  weight  and  relaxed  transportability  requirements,  in  what  may  be  an  effort  to  shield 
the  program  from  lawmakers  seeking  to  terminate  it.”  Tony  Bertuca,  “Army  and  Marines  Agree  on  Requirements 
Change  for  Embattled  JLTV,”  InsideDefense.  com:  Inside  the  Navy,  September  30,  201 1 . 

29  “JPALS  Increment  1A  Selected  Acquisition  Report,”  Defense  Acquisition  Management  Information  Retrieval 
(DAMIR),  December  31,  2011,  p.  4. 
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JPALS,  is  separated  into  two  phases:  Increment  1A  ship  based  systems  and  Incre¬ 
ment  IB  aircraft  integration.30 

The  program’s  last  major  milestone  awarded  was  Milestone  B  on  July  14,  2008.  Table 
4.4  provides  the  major  framing  assumptions  for  the  JPALS  program. 

The  first  major  faming  assumption  examined  is  a  technological  one  stating  that 
incremental  development  will  lower  risk  and  COTS/GOTS  hardware  and  software 
will  lower  costs.  Incremental  development  and  the  use  of  COTS/GOTS  are  strate¬ 
gies  that  are  frequently  combined  in  acquisition  of  weapons  systems.  JPALS  is  using 
incremental  development  (i.e.,  an  evolutionary  acquisition  approach)  to  lower  risk.31 
Evolutionary  acquisition  is  used  in  defense  acquisition  as  a  way  to  fulfill  warfighter 
needs  incrementally  as  technology  matures.  It  also  allows  programs  to  put  off  fulfilling 
some  of  the  requirements  until  cost  and  technology  are  better  understood.  In  the  case 
of  JPALS,  the  current  ACAT  ID  encompasses  all  seven  increments,  but  the  program 
is  primarily  focusing  on  the  Navy  requirements  of  landing  on  a  carrier  in  Increment  1. 
The  program  office  has  started  preparing  the  technology  for  Increment  2. 

The  program  is  using  an  incremental  acquisition  strategy  to  manage  both  risks 
and  costs.  In  addition,  the  program  is  leveraging  COTS/GOTS  hardware  and  software 
also  as  a  cost  savings  tool.  The  program  uses  mature  technologies:  P(Y)-code  GPS, 
anti-jam  GPS,  and  military  radios  for  data  link.32  According  to  GAO: 

JPALS  is  primarily  a  software  development  effort  but  also  includes  com¬ 
mercial  hardware  components.  .  .  .  The  JPALS  program  began  development 
in  July  2008  with  two  critical  technologies — the  geometry  extra  redundant 
almost  fixed  solution  and  the  vertical  protection  level/lateral  protection  level — 
nearing  maturity.  Program  officials  expect  both  critical  technologies  to  be  mature 
and  demonstrated  in  a  realistic  environment  by  the  JPALS  production  decision  in 

Table  4.4 

JPALS  Framing  Assumptions 


Technological 


Management,  Incentives, 

and  Program  Structures  Mission  Requirements 


Incremental  development  will  Navy  is  the  best  service  to  lead  System  is  suitable  for  a  variety 

lower  risk,  while  COTS/GOTS  acquisition  with  the  ordering  of  of  air  vehicles 

hardware  and  software  will  lower  the  increments  driven  by  the  Navy's 
costs  more  demanding  requirements 


30  “JPALS  Increment  1A  Selected  Acquisition  Report,”  2011,  p.  4. 

31  JPALS  Program  Office,  “Joint  Precision  Approach  and  Landing  System  (JPALS):  Acquisition  Strategy  in  Sup¬ 
port  of  Milestone  B  for  JPALS  Increment  1  A,”  Patuxent  River,  Md.:  Naval  Air  Systems  Command,  June  2008, 
p.  2.  Not  available  to  the  general  public. 

32  Jeff  Stevens  and  Brian  Pierc  z.  Joint  Precision  Approach  and  Landing  System  (JPALS),  Land-Based  JPALS  Tech¬ 
nical  Overvieiv,  JPALS  Program  Office,  AFCEA  CNS/ATM  2011,  June  15,  2011,  p.  25. 
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2013.  While  JPALS  utilizes  existing  commercial  components  for  most  of  its  hard¬ 
ware,  its  functionality  will  be  enabled  by  over  700,000  lines  of  software  code.  The 
program  plans  to  rely  heavily  on  reused  code  with  77  percent  of  the  program’s  total 
lines  of  code  expected  to  be  reused.  If  less  software  is  reused  than  originally  esti¬ 
mated,  the  potential  consequences  are  longer  development  time  and  greater  cost.33 

In  addition,  the  program  is  also  using  analogous  manufacturing  processes/costs, 
open  architecture,  and  simple  retrofitting  of  existing  aircraft. 

JPALS  will  reduce  the  number  of  required  navigation  systems  onboard  aircraft  car¬ 
riers,  cutting  cost  and  weight,  and  freeing  valuable  avionics  space.  .  .  .  JPALS  will 
eliminate  expensive  legacy  systems,  yielding  large  infrastructure,  operation,  and 
support  cost  savings  .  .  .  JPALS  simplicity  and  standardization  will  reduce  pilot 
training  requirements  and  will  lower  training  costs.34 

The  second  major  framing  assumption  falls  under  the  category  of  “management, 
incentives,  and  program  structures.”  This  assumption  says  that  the  Navy  will  be  the 
best  service  to  lead  acquisition.  On  July  21,  2007,  the  Navy  was  designated  as  the  lead 
service  for  the  EMD  phase.35  The  Navy  took  over  from  the  Air  Force,  which  led  the 
program  to  Milestone  B.  The  Navy’s  requirements  for  the  sea-based  system  begin  with 
Increment  1  and  are  refined  in  subsequent  increments.  The  fact  that  the  Navy  has  the 
first  increment  and  has  the  more  difficult  technical  problem  makes  the  Navy’s  role  as 
lead  a  natural  fit.  Other  services  play  a  critical  role  in  the  program  for  the  other  incre¬ 
ments  after  Increment  1.  Budgetary  concerns  have  forced  both  the  Air  Force  and  the 
Army  to  take  a  serious  look  at  their  role  in  the  system.  Without  the  Air  Force  and  Army 
involvement,  this  program  would  not  achieve  its  goal  of  supporting  “all  Department  of 
Defense  (DoD)  Land  and  Sea  Based  applications.”  See  Figure  4.1  for  an  understanding 
of  how  the  program  is  structured. 

The  final  major  framing  assumption  is  that  the  system  is  suitable  for  all  types  of 
air  vehicles.  This  is  a  critical  framing  assumption  because  if  the  program  finds  that 
integration  of  a  variety  of  vehicles  from  the  different  services  is  harder  than  anticipated, 
then  this  will  likely  produce  cost  and  schedule  problems  as  was  the  case  with  the  C-130 
AMP  program.  According  to  an  open  source  briefing  by  the  JPALS  program  office, 
JPALS  “aircraft  system  design  flexibility  [is]  extremely  important  .  .  .  [there  is  a]  large 


33  U.S.  Government  Accountability  Office,  Defense  Acquisitions:  Assessments  of  Selected  Weapon  Programs,  Wash¬ 
ington,  D.C.,  GAO-1 1-233SP,  March  2011a,  pp.  89-90. 

34  Defense  Technical  Information  Center,  “Defense  Standardization  Program  Case  Study:  Joint  Precision 
Approach  and  Landing  System,”  date  unknown. 

35  U.S.  Department  of  Defense,  “Exhibit  R-2,  RDT&E  Budget  Item  Justification:  PB  2012  Air  Force:  JPALS,” 
February  2011,  p.  1. 
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Figure  4.1 

JPALS  Tri-Service  Structure 


SOURCE:  Col  Jimmie  Schuman,  Laura  Dionne,  Steele  Kenyon,  Chuck  Churchward,  Capt  Michael  Scales, 
Sandy  Frey,  Ben  Brandt,  Tin  Ying,  Jeff  Stevens,  and  Tammy  Place,  "Joint  Precision  Approach  and  Landing 
System  (JPALS):  Land-Based  JPALS  Industry  Day,"  JPALS  Program  Office,  December  8-10,  2009,  p.  11. 
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variation  in  aircraft  architectures.”36  In  addition,  the  JPALS  program  has  also  stressed 
the  importance  of  worldwide  and  civil  interoperability  in  defense  budget  documents: 

Because  a  cornerstone  of  the  JPALS  implementation  strategy  is  worldwide  and  civil 
interoperability,  JPALS  must  harmonize  with  U.S.  and  International  Civil  Global 
Navigation  Satellite  Systems.  This  is  being  accomplished  through  participation  in 
the  development  testing,  and  implementation  of  international  standards  through 
the  North  American  Treaty  Organization  (NATO)  the  International  Civil  Avia¬ 
tion  Organization  (ICAO).  Interoperability  of  the  JPALS  ground  systems  with  all 
military  and  civil  aircraft  is  a  key  aspect  of  the  planned  system.  Military  aircraft 
must  have  worldwide  access  to  civil  and  military  airfields/air  stations/operating 
locations  in  benign  and  hostile  (jamming)  environments.  The  JPALS  Land-based 
Increment  2  system  will  provide  a  civil  interoperable  capability  and  also  a  military 
interoperable  encrypted,  jam-resistant  capability.37 


36  Schuman  et  al.,  2009,  p.  27. 

37  U.S.  Department  of  Defense,  2011,  p.  1. 
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In  addition  to  the  above  major  framing  assumptions  it  is  also  worth  noting  that 
JPALS  relies  heavily  on  GPS,  so  a  minor  assumption  is  that  the  GPS  constellation  will 
be  robust.  According  to  the  JPALS  program  office,  the  program  uses  a  differential 
GPS -based  system  that  comprises  ground  systems  and  aircraft  systems.  The  program 
also  uses  a  JPALS -specific  data  link  and  existing  GPS  satellites.38 

Littoral  Combat  Ship  Modules 

The  LCS  modules  is  a  Navy  mission  systems  program  that  is  transitioning  from  pre- 
MDAP  to  MDAP  status.  This  major  acquisition  program  is  a  recent  spinoff  of  the 
main  LCS  program  that  includes  the  ships.  “LCS  operates  with  focused-mission  pack¬ 
ages  that  deploy  manned  and  unmanned  vehicles  to  execute  a  variety  of  missions, 
including  littoral  anti-submarine  warfare  (ASW),  surface  warfare  (SUW),  and  mine 
countermeasures  (MCM).”39  The  mission  modules  are  being  acquired  using  an  evo¬ 
lutionary  acquisition  strategy,  and  the  LCS  program  office  is  incrementally  adding 
mature  mission  systems.40  The  mission  modules  will  also  be  flexible  and  scalable.  The 
program  office  has  chosen  to  use  open  architecture,  which  may  enable  rapid  inser¬ 
tion  of  new  technologies.  The  program’s  last  milestone  awarded  was  Milestone  A  on 
May  27,  2004,  and  underwent  a  Milestone  A  update  in  FY2009  as  part  of  the  Littoral 
Combat  Systems  program. 

The  LCS  mission  modules  program  has  a  longer  list  of  framing  assumptions  than 
other  programs  (see  Table  4.5).  We  chose  the  major  assumptions  to  discuss  first.  The 
first  major  framing  assumption  is  a  technological  one,  which  says  that  the  sea  frames 
and  modules  can  be  independently  developed.  “Mission  package  procurement  and 
delivery  are  aligned  with  the  ship  delivery  schedule,  mission  area  demand  signal  from 
the  combatant  commanders,  and  the  retirement  of  legacy  platforms.  This  means  that 
64  interchangeable  mission  packages  will  be  available  for  use  among  the  55  ships  of 
the  LCS  class  to  support  global  warfighting  and  peacetime  presence  requirements.”41 
If  the  schedules  between  the  two  coincide  and  neither  gets  behind  schedule,  then  this 
strategy  can  succeed. 

This  main  framing  assumption  has  some  relevant  subordinate  points.  First,  the 
program  is  using  spiral  or  incremental  development  that  will  lower  risk  and  possibly 
costs.  “An  incremental  development  approach  to  delivering  capability  allows  the  con¬ 
tinued  insertion  of  mature  capabilities  throughout  the  life  of  the  program  without  the 
need  for  modifications  to  the  sea  frames.  Future  mission  package  increments  will  be 
considered  when  joint  warfighting  objectives  or  changing  threats  create  new  opera- 


38  Schuman  et  al.,  2009,  p.  10. 

39  U.S.  Department  of  Defense,  “Exhibit  R-2,  Exhibit  R-2A,  RDT&E  Project  Justification:  PB  2013  Navy:  LCS 
Mission  Package  Development,”  February  2012,  p.  39. 

40  U.S.  Department  of  Defense,  2012,  p.  29. 

41  U.S.  Department  of  Defense,  2012,  p.  15. 
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Table  4.5 

LCS  Modules  Framing  Assumptions 


Technological 


Management,  Incentives, 

and  Program  Structures  Mission  Requirements 


Sea  frames  and  modules  can  be 
independently  developed: 

Spiral/incremental  development 
will  lower  program  risk 

Ability  to  successfully  test 
modules  on  other  ship 
platforms 

New  capabilities  can  be  easily 
added  to  meet  future  needs 


A  new,  open  business  model  approach  The  Navy  will  be  willing 
will  allow  for  independent  develop-  to  drop  requirements  (in  a 
ment  of  sea  frames  and  modules:  spiral  development 

context)  to  keep  to 

The  Navy  can  leverage  PARMs  and  schedule 
other  programs  to  field  a  capability 

Open  architecture/commercial 
practices  will  facilitate  development 
and  rapid  updates 

The  government  is  suited  to  act  as 
system  integrator 


RDA  can  be  the  program  focal  point 
(4  PMs,  3  PEOs,  2  SYSCOMS)  for  the 
program 


tional  capability  requirements  that  cannot  be  met  by  current  mission  package  designs, 
or  when  new  technological  opportunities  allow  significant  progress  toward  deliver¬ 
ing  cost  effective,  enhanced  capabilities.  Future  mission  module  increments  can  be 
tested,  constructed,  and  incorporated  into  existing  mission  packages,  one  of  the  most 
important  benefits  of  LCS  modular  design.”42  In  addition,  this  incremental  develop¬ 
ment  strategy  will  allow  new  capabilities  to  be  added  easily,  which  is  another  minor 
assumption.  Also,  the  program  is  relying  on  its  ability  to  test  modules  successfully  on 
other  ship  platforms.  This  assumption  is  important,  because  if  ships  are  not  available 
for  testing  because  of  operational  constraints,  then  this  may  delay  testing  and  overall 
schedule. 

The  second  major  framing  assumption  falls  under  the  management,  incentives, 
and  program  structures  category.  For  the  LCS  mission  modules  program,  a  new,  open 
business  model  approach  allows  for  independent  development  of  sea  frames  and  mod¬ 
ules.  According  to  PEO  LCS:  “Naval  Open  Architecture  .  .  .  Increases  opportunities 
for  innovation  .  .  .  Facilitates  rapid  technology  insertion  .  .  .  [and]  Reduces  mainte¬ 
nance  constraints.”43  The  use  of  an  open  business  model  also  allows  for  better  manage¬ 
ment  of  technical  data  management,  which  is  a  key  to  reusing  technical  data  for  the 


42  U.S.  Department  of  Defense,  2012,  p.  15. 

43  Megan  Cramer,  “Program  Executive  Office  Littoral  Combat  Ships  (PEO  LCS)  Science  and  Technology  Over¬ 
view,”  PEO  LCS,  August  31,  2011,  p.  18. 
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program.44  In  the  case  of  the  LCS  program,  this  concept  will  be  successful  as  long  as 
the  time  lines  for  development  of  both  do  not  significantly  divert. 

There  are  several  other  points  to  make  regarding  this  framing  assumption.  First, 
the  program  will  successfully  leverage  PARMs  and  other  programs.  “The  modules  lever¬ 
age  considerable  amounts  of  technology  from  existing  POR  [programs  of  record].”45 
Also,  the  “Warfare  Centers  are  able  to  rapidly  mature  prototype  concepts  and  support 
transition  to  production  via  existing  contract  vehicles.”46  By  using  Navy  resources  out¬ 
side  Program  Manager,  Ship  (PMS)  420,  the  Navy  believes  it  will  be  able  to  handle 
the  lead  system  integrator  duties  more  effectively.  See  Figure  4.2  for  the  various  Navy 
PARMs  involved  in  the  LCS  mission  modules  program. 

In  addition,  the  Navy  will  benefit  from  using  open  architecture  and  commercial 
practices.  For  example,  the  Navy  is  implementing  commonality  across  disperse  prod¬ 
uct  lines  as  a  way  to  reduce  both  cost  and  schedule.47  The  Navy  will  also  motivate  the 
Navy  labs  to  design  toward  producible  designs  as  a  way  to  minimize  redesign.48  This 
program  also  assumes  that  the  government  is  suited  to  act  as  system  integrator.  In  this 
program,  PMS  420’s  acquisition  approach  relies  heavily  on  using  the  Navy  Warfare 
Centers  as  lead  system  integrators  (LSIs).49  “Using  the  Warfare  Centers  as  LSIs  gives 
the  freedom  to  implement  commonality  without  multiple  contract  modifications.”  50 
Finally,  the  ASN(RDA)  will  successfully  be  the  program  focal  point,  since  there  are 
four  PMs,  three  PEOs,  and  two  SYSCOMS  involved  in  managing  portions  of  the 
program.  Effective  July  11,  2011,  the  offices  that  manage  the  sea  frame  and  mission 
module  programs  were  combined  under  PEO  LCS — largely  simplifying  this  issue. 
Before  this  reorganization,  the  ASN  (RDA)  established  the  LCS  program  office  on 
February  1,  2002.  On  March  17,  2003,  PEO  (Ships)  was  given  overall  responsibility 
for  program  management,  PEO  (LMW)  for  mission  modules,  and  PEO  (IWS)  for 
warfare  system  development  and  integration.  In  addition,  on  May  5,  2011,  the  ASN 
(RDA)  directed  the  formation  of  PEO  LCS,  which  included  the  following  six  offices: 


44  Robin  Kime  and  Wayne  Gafford,  “LCS  Mission  Modules  Program:  Training  Strategy  Increasing  Modularity 
for  Maximum  Adaptability;  Brief  for  Implementation  Fest  2010,”  PEO  LMW  and  PMS  420,  August  10,  2010, 
p.  3. 

45  Lance  Harper,  Art  Van  Nostrand,  Mike  Pennock,  and  William  Algoso,  “System  of  Systems  Technology  Anal¬ 
ysis  and  Selection  Methodology  2010  NDIA  Systems  Engineering  Conference,”  Northrop  Grumman,  October 

28,  2010,  p.  6. 

46  Cecil  Whitfield,  Richard  Volkert,  and  Carly  Jackson,  “Navy  Warfare  Centers  as  Lead  System  Integrators:  Les¬ 
sons  Learned  from  Mission  Module  Development  13th  Annual  NDIA  Systems  Engineering  Conference,  PMS 
420  LCS  Mission  Modules,  October  27,  2010,  p.  8. 

47  Whitfield,  Volkert,  and  Jackson,  2010,  p.  17. 

48  Whitfield,  Volkert,  and  Jackson,  2010,  p.  17. 

49  Whitfield,  Volkert,  and  Jackson,  2010,  p.  7. 

50  Whitfield,  Volkert,  and  Jackson,  2010,  p.  11. 
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Figure  4.2 

PMS  420's  Use  of  Lead  System  Integrators  and  Other  Navy  PARMs 
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•  LCS  program  (PMS  501) 

•  LCS  mission  modules  (PMS  420) 

•  remote  minehunting  system  (PMS  403) 

•  unmanned  maritime  systems  (PMS  406) 

•  mine  warfare  (PMS  495) 

•  fleet  introduction  (PMS  505). 51 

This  framing  assumption  is  a  critical  one,  because  so  many  parties  are  involved  in 
the  management  of  the  LCS  system.  Without  proper  communication  among  the  vari¬ 
ous  pieces  of  the  program,  this  framing  assumption  could  be  problematic. 


5*  RDML  Jim  Murdoch,  USN,  Expeditionary  Warfare  Systems  Acquisition  Challenges,  Washington,  D.C.:  U.S. 
Navy,  PEO  LCS,  October  26,  2011,  p.  4. 
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The  third  major  framing  assumption  is  the  willingness  of  the  Navy  to  drop 
requirements  (in  spiral  context)  to  keep  to  schedule.  The  mission  modules  program  set 
a  goal  of  to  “implement  effective  requirements  management  and  drive  towards  com¬ 
monality  early  in  the  design  process.”52  This  is  being  done  through  the  establishment 
of  a  cross  organization  requirements  database.  Also,  staying  on  schedule  is  important 
in  this  program’s  acquisition  strategy,  because  the  sea  frame  and  modules  are  being 
independently  developed,  yet  the  modules  must  be  available  when  needed. 

The  LCS  mission  modules  program  had  two  other  secondary  assumptions  that 
are  important  to  note  but  would  not  be  considered  the  primary  framing  assumptions 
for  the  program.  However,  both  secondary  assumptions  relate  to  the  decoupling  of  the 
mission  modules  from  the  sea  frames.  The  first  of  these  secondary  assumptions  is  that 
any  organizational-,  intermediate-,  and  depot-level  maintenance  will  be  done  success¬ 
fully  at  a  common  facility,  i.e.,  “a  hub  for  all  in-service  mission  modules.”  Thus,  main¬ 
tenance  of  the  modules  is  independent  of  the  ship’s  maintenance  and  not  done  at  a  tra¬ 
ditional  ship  maintenance  facility.  The  objectives  of  this  facility  include  the  following: 

•  Provide  distance  support  for  deployed  mission  modules. 

•  Configure  certified  deployable  assets. 

•  Troubleshoot  and  repair. 

•  Conduct  system  operability  tests. 

•  Conduct  inventory  management/visibility. 

•  Validate  ready-for-use  status  of  the  mission  package  (MP). 

•  Perform  packaging,  handling,  storage,  and  transportation. 

•  Ensure  material  shelf  life. 

•  Ensure  that  authorized  spares  are  onboard. 

•  Replenish  spares  and  consumables. 

•  Expedite  parts  requests  as  required. 

•  Arrange  transportation  of  mission  modules. 

•  Arrange  embark  and  debark  services.53 

The  use  of  a  common  facility  can  be  advantageous  if  the  mission  modules  can 
be  rotated  so  that  there  are  no  operational  deficiencies.  Also,  as  long  as  minor  main¬ 
tenance  can  be  done  offsite,  then  this  would  not  require  mission  modules  to  be  trans¬ 
ported  from  all  over  the  world  to  this  facility. 

The  other  secondary  assumption  is  that  the  mission  modules  will  be  funded  with 
a  different  account — Other  Procurement,  Navy — rather  than  using  Shipbuilding  and 
Conversion,  Navy.  This  different  funding  approach  for  the  modules  formally  discon- 


52  Whitfield,  Volkert,  and  Jackson,  2010,  p.  17. 

53  Kime  and  GafFord,  2010,  p.  8. 


Framing  Assumptions  101 


nects  the  deliveries  and  quantities  of  modules  with  that  for  the  ships.  It  also  implicitly 
declares  that  a  module  is  not  an  inherent  part  of  the  sea  frame. 

Space  Fence 

Space  Fence  is  an  Air  Force,  pre-MDAP  acquisition  program  that  seeks  to  provide  a 
radar  system  to  replace  the  Air  Force  Space  Surveillance  System  Very  Fligh  Frequency 
(AFSSS  VHF)  Fence  radar  that  currently  detects  orbiting  space  objects.  Its  S-band 
radar  will  have  modern,  net-centric  architecture  that  is  able  to  detect  much  smaller 
objects  in  low  earth  orbit  and  medium  earth  orbit  with  greater  accuracy  and  timeliness 
to  meet  warfighter  requirements  for  space  situational  awareness  (SSA).  Two  radar  sites 
are  to  be  built  outside  the  continental  United  States. 

Despite  identified  technological  risks  of  “defining  software  requirements”  and 
“identify  interoperability  and  integrated  architectures,”  the  Space  Fence  acquisition 
strategy  determined  “that  the  U.S.  technology  and  industrial  base  is  more  than  ade¬ 
quate  to  develop,  produce,  maintain,  and  support  the  system”  for  SSA  at  a  far  greater 
magnitude  of  objects  than  attempted  before.54  Two  framing  assumptions  are  embed¬ 
ded  here  (see  Table  4.6).  The  first  is  that  the  operational  capability  can  be  achieved 
(successfully)  despite  having  some  immature  technologies.  The  second  is  that  one  can 
scale  the  technology  to  track  an  order  of  magnitude  greater  number  of  objects  that  is 
currently  done.  In  addition  to  assumptions  about  technological  feasibility,  the  program 
also  assumes  that  competitive  prototyping  will  reduce  the  risk  and  cost  as  well  as  be 
representative  of  the  final  integration  challenge.  Lockheed  Martin  and  Raytheon  were 
each  tasked  to  create  a  Space  Fence  preliminary  design  with  prototype  designs  with  a 
Technology  Readiness  Level  6,  and  all  five  critical  technologies  identified  by  the  pro¬ 
gram  office  were  considered  “immature”  at  the  initial  stages.55  Although  the  risks  of 


Table  4.6 

Space  Fence  Framing  Assumptions 


Technological 


Management,  Incentives, 

and  Program  Structures  Mission  Requirements 


Capability  is  achievable  despite  some  A  block  approach  is  a  more 
immature  technologies  at  the  outset  effective  acquisition  strategy 


Can  scale  technology  to  track  an  order 
of  magnitude  greater  number  of 
objects  (radar  components,  software 
interoperability) 


Legal,  diplomatic,  and  political 
issues  with  site  decisions  can 
be  resolved  easily  and  will  not 
delay  the  program 


Minimal  manpower  will  be 
required  to  operate  and 
support  the  system 

Two  instead  of  three  sites 
will  be  sufficient  to  meet 
operational  needs 


Competitive  prototyping  will  reduce 
risk  and  cost 


54  “Acquisition  Strategy  for  the  Space  Fence  Program,”  Space  Control  Sensors  Division,  January  5,  2009. 

55  U.S.  Government  Accountability  Office,  DOD  Faces  Challenges  in  Fully  Realizing  Benefits  of  Satellite  Acquisi¬ 
tion  Improvements,  Washington,  D.C.:  Government  Accountability  Office,  GAO-12-563T,  March  21,  2012. 
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final  integration  and  scalability  challenge  after  the  prototype  selection  are  understood, 
“there  are  no  major  concerns  in  dealing  with  these  issues  in  the  future,”  according  to 
the  program  manager.56 

Under  the  management,  incentives,  and  program  structures  framing  assump¬ 
tions,  it  was  assumed  that  “the  block  approach  will  deliver  Space  Fence  capabilities 
that  follow  the  principles  of  time-certain  capability/development  and  consider  user 
needs  and  required  delivery  dates,  technology  maturity,  program  risk,  and  fiscal 
constraints.”57  In  addition  to  the  acquisition  plan,  although  program  managers  were 
aware  that  “legal,  diplomatic,  and  political  considerations  could  impact  site  decisions,” 
it  appears  that  assumptions  concerning  the  ease  and  implications  of  obtaining  host 
nation  agreements  for  the  radar  sites,  because  of  the  unknowns  at  the  time,  framed 
costing  and  scheduling  estimations.58 

Another  important  assumption  (although  not  a  framing  assumption  because  it  is 
not  unique)  was  that  the  Air  Force  could  overcome  the  problems  it  had  on  recent  space 
program  acquisitions  through  the  “Acquisition  Improvement  Plan”: 

The  Secretary  of  rhe  Air  Force  and  Chief  of  Staff  of  the  Air  Force  issued  the  Acqui¬ 
sition  Improvement  Plan  to  recapture  acquisition  excellence  by  rebuilding  an  Air 
Force  acquisition  culture  that  delivers  products  and  services  as  promised — on  time, 
within  budget,  and  in  compliance  with  all  laws,  policies,  and  regulations.  The  plan 
consists  of  five  initiatives:  (1)  revitalize  the  Air  Force  acquisition  workforce,  (2) 
improve  the  requirements  generation  process,  (3)  instill  budget  and  financial  disci¬ 
pline,  (4)  improve  major  Air  Force  systems  source  selections,  and  (5)  establish  clear 
lines  of  authority  and  accountability  within  acquisition  organizations.59 

In  terms  of  longer-term  sustainment,  the  total  SSA  family  of  systems,  of  which 
Space  Fence  is  a  component,  is  assumed  able  to  help  the  Joint  Space  Operations  Center, 
“do  its  job  without  a  huge  increase  in  personnel.”60  In  addition  to  a  minimal  need 
of  manpower  to  operate  and  support  the  system,  the  Space  Fence  program  assumed 
that  its  mission  would  require  “the  capability  to  process  satellite  numbers  up  to  seven 
digits”  as  the  number  of  objects  in  space  from  civil,  military,  and  commercial  use  will 


56  Telephone  interview  with  Linda  Haines,  Space  Fence  Program  Manager,  June  2,  2011;  Titus  Ledbetter  III, 
“Air  Force  Confident  Space  Fence  Contractors  Will  Resolve  Program  Risks,”  InsideDefense.com:  Inside  the  Air 
Force ,  June  2,  2011. 

57  “Space  Fence  Acquisition  Strategy,”  February  23,  2009. 

58  Air  Force  Space  Command,  “Enabling  Concept  for  the  Space  Fence,”  August  4,  2008,  p.  6. 

59  Cristina  T.  Chaplain,  Space  Acquisitions:  DOD  Delivering  New  Generations  of  Satellites,  but  Space  System 
Acquisition  Challenges  Remain ,  Washington,  D.C.:  U.S.  Government  Accountability  Office,  GAO  11-590T, 
May  11,  2011. 

60  As  stated  by  the  Air  Force’s  top  intelligence,  surveillance,  and  reconnaissance  official  at  the  time,  Lt.  Gen. 
David  Deptula.  Jason  Simpson,  “Payton:  New  Missions  System  Will  Mitigate  SSA  Information  Overload,” 
InsideDefense.com:  Inside  the  Air  Force,  November  6,  2009. 
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continue  to  grow.61  The  final  framing  assumption  concerns  the  number  of  radar  sites. 
The  Space  Fence  Enabling  Concept  Plan62  stated  that,  “The  Space  Fence  will  consist 
of  three  radars  globally  dispersed  [one  in  Continental  United  States  (CONUS)  and 
two  outside  of  the  Continental  United  States  (OCONUS)]  to  ensure  the  most  effective 
combined  [Space  Surveillance  Network]  coverage.”  More  recent  documents  describe 
the  scope  as  only  two  sites.63  A  GAO  report  suggests  that  the  third  site  might  be 
dropped  for  cost-effectiveness  reasons.64 

Framing  Assumptions  Based  on  Prior  RAND  Root  Cause  Analyses 

RAND  has  conducted  a  number  of  RCAs  on  programs  that  have  breached  Nunn- 
McCurdy  thresholds.65  We  abstracted  a  number  of  framing  assumptions  that  those 
reports  identify  and  organized  them  into  our  three  categories  (as  shown  in  Table  4.7). 

Table  4.7 

Framing  Assumptions  from  Prior  RCAs 


Technical 


Management/Program 

Structures  Mission  Requirements 


Reliance  on  commercial 
technology  increases  likelihood 
of  achieving  technical  program 
goals  (P-8A,  WGS,  ERP) 

Successful  subsystem  testing 
predicts  overall  technical 
program  success  (JSF,  Excalibur, 
AB3,  DDG-1000). 

Simulations  can  substitute  for  or 
reduce  full-scale  testing  (JTRS, 
JSF) 

New  testing/manufacturing 
approaches  can  reduce  historic 
number  of  test  articles  (JTRS,  JSF, 
P-8A) 


Reliance  on  commercial  initiative/  Low  possibility  for  emergence  of  a 
standards  insulates  the  program  substitute  good  (JTRS) 


from  risk  (P-8A,  WGS,  ERP) 

Threats  to  program  funding  are 
an  incentive  to  manage  program 
risk  (ERP,  DDG-1000,  JSF) 

Management  changes  do  not 
detract  from  program  outlook 
(JTRS,  JSF) 

Program  subcomponent/ 
integration  risks  not  a  geometric 
risk  function  (JSF,  AB3,  DDG-1000, 
JTRS,  Excalibur) 


Constancy  with  joint  capability 
requirements  (JSF) 

Reliance  on  commercial  technology 
shields  program  from  risk  and 
allows  program  to  adapt  quicker  to 
changing  operational  requirements 
(P-8A) 


SOURCES:  Blickstein  et  a  I .,  2011,  2012,  and  Irv  Blickstein,  Chelsa  Kahoi  Duran,  Daniel  Gonzales,  Jennifer 
Lamping  Lewis,  Charles  Nemfakos,  Jessie  Riposo,  Rena  Rudavsky,  Jerry  M.  Sollinger,  and  Daniel  Tremblay, 
Root  Cause  Analyses  of  Nunn-McCurdy  Breaches  Volume  3,  Joint  Tactical  Radio  System,  P-8A  Poseidon, 
and  Global  Hawk  Modifications,  Santa  Monica,  Calif.:  RAND  Corporation,  MG-1171/3,  2013.  Not  available 
to  the  general  public. 


61  Air  Force  Space  Command,  2008,  p.  6. 

62  Air  Force  Space  Command,  2008. 

63  “Space  Fence  Contractors  Complete  PDRs,  Praise  USAF  Review  Process,”  Inside  the  Air  Force,  March  9, 

2012. 

64  U.S.  Government  Accountability  Office,  Space  Acquisitions:  Developing  and  Oversight  Challenges  in  Delivering 
Improved  Space  Situational  Awareness  Capabilities,  GAO-11-545,  May  2011b. 

65  See  Blickstein  et  ah,  2011,  2012,  and  2013. 
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It  is  important  to  recognize  that  those  reports  are  focused  on  why  the  programs  had 
cost  and  schedule  problems.  So,  their  emphasis  is  on  what  went  wrong.  The  framing 
assumptions  identified  are  ones  that  turned  out  to  be  incorrect  or  invalid.  We  did  not 
do  a  comprehensive  literature  review  for  these  programs. 


Observations 

In  this  chapter,  we  have  explored  the  definition  and  application  of  framing  assump¬ 
tions  to  defense  acquisition  planning.  Framing  assumptions  have  the  following 
characteristics: 

•  They  drive  multiple  program  consequences  (with  respect  to  cost,  schedule 
performance). 

•  They  are  specific  to  the  nature  of  the  program  effort. 

We  also  found  it  practical  to  define  and  review  framing  assumptions,  even  early 
in  a  program’s  life  cycle.  We  were  able  to  establish  framing  assumptions  for  all  the 
programs  despite  having  limited  access  to  program  information.  There  are  some  useful 
observations  in  using  framing  assumptions  in  a  risk  management  role: 

•  The  assumptions  should  be  kept  at  a  high  level  and  not  focus  on  minutiae  to  be 
more  useful  as  a  management  tool.  About  three  to  five  is  a  good  number  to  track 
for  most  programs.  However,  complicated  programs  or  those  with  a  rich  history 
may  become  challenging  to  reduce  to  a  few  key  assumptions,  as  the  importance 
of  the  assumptions  is  difficult  to  know  ahead  of  time. 

•  As  a  program  moves  through  the  acquisition  process,  framing  assumptions  may 
change  or  new  ones  emerge.  Therefore,  it  is  necessary  to  review  and  update  fram¬ 
ing  assumptions  periodically. 

•  For  programs  early  in  the  process,  it  is  relatively  easy  to  identify  a  narrow  set  of 
explicit  assumptions.  However,  it  is  difficult  to  uncover  implicit  ones.  For  mature 
programs,  it  is  more  complicated  to  identify  a  core  set  of  framing  assumptions 
because  of  the  wealth  of,  and  potentially  conflicting,  information. 

•  It  is  easier  to  recognize  after-the-fact  failed  assumptions  than  it  is  to  recognize 
good  ones. 

How  successful  the  use  of  framing  assumptions  can  be  in  managing  risk  remains 
to  be  seen.  A  potentially  useful  extension  of  this  work  would  be  to  examine  fram¬ 
ing  assumptions  on  a  broader  range  of  programs  to  see  if  particular  assumptions  are 
problematic.  Overall,  this  analysis  suggests  that  framing  assumptions  are  a  potentially 
useful  tool  for  tracking  risks  and  that  early  identification  of  them  in  programs  could 
be  an  important  part  of  risk  mitigation. 


CHAPTER  FIVE 

Conclusions 


PM  Tenure 

It  is  difficult  to  assess  whether  policies  implemented  to  foster  longer  tenures  have  been 
successful  judging  by  previous  data  and  studies  and  current  data.  Given  that  the  same 
themes  have  reoccurred  in  policies  over  the  past  40  years,  the  intent  of  these  policies 
may  not  have  been  achieved.  Furthermore,  an  enforcement  mechanism  has  not  been 
readily  apparent  over  time.  A  fundamental  conflict  exists  between  what  military  offi¬ 
cers  need  to  do  to  be  promoted  and  their  tenure  as  PMs.  Unless  these  two  objectives 
are  connected  so  that  lengthy  tenure  in  a  program  can  be  advantageous  for  promotion, 
it  is  unlikely  that  these  tenure  policies  will  consistently  yield  positive  results. 

PMs  who  started  in  2006  or  2007  had  about  the  same  probability  of  remaining 
in  their  position  for  at  least  two  years  but  a  greater  probability  of  staying  for  three  or 
more  years  than  PMs  who  started  in  2005  or  earlier.  PMs  starting  in  2008  or  later  had 
about  an  87  percent  chance  of  remaining  in  their  positions  for  two  or  more  years  and 
about  a  50  percent  chance  of  remaining  for  three  or  more  years,  suggesting  an  increase 
in  PM  tenure  after  the  second  policy  change. 

By  taking  into  account  closed  PM  tenure  periods,  we  found  that  PM  tenure  aver¬ 
ages  33  months.  This  result  is  much  higher  than  the  GAO’s  2007  figure  of  17.2  months 
(based  on  a  sample  of  39  programs).  Fiowever,  our  figure  reflects  a  larger  set  of  pro¬ 
grams  covering  more  years  and  does  not  include  any  open  periods. 

In  conclusion,  our  analysis  was  able  to  quantify  PM  tenure  using  current  data 
but  could  not  definitely  say  whether  recent  policies  regarding  PM  tenure  have  had  any 
positive  effect  on  lengthening  tenure  over  the  past  several  years,  because  there  were  still 
too  many  open  tenure  periods  at  the  time  of  data  collection. 


ACAT  II  Oversight 

Our  analysis  suggests  that  ACAT  II  programs  perform  about  as  well  as  MDAPs  overall. 
Once  they  have  entered  into  production,  programs  perform  reasonably  effectively,  with 
some  instability  in  unit  cost  and  annual  quantity  procured  but  few  instances  of  rapid, 
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long-term  cost  growth  or  significant  slippage  in  schedule.  In  both  cases,  cost  growth 
and  instability  in  quantity  procured  are  associated  with  modernizations,  contingency 
operations,  and  accelerated  time  lines  but  tend  to  be  limited  in  size  and  often  duration. 

As  to  whether  the  PARCA  office  should  issue  guidance  for  ACAT  II  programs, 
we  recommend  that  it  not  do  so  at  this  time.  The  performance  of  the  programs  indi¬ 
cates  that  they  are  doing  reasonably  well,  and  any  new  reporting  requirements  would 
place  additional  burdens  on  programs  that  already  have  much  to  do.  Absent  a  clear 
need  for  such  guidance,  the  PARCA  office  should  not  impose  additional  requirements 
but  should  continue  to  monitor  programs  to  ensure  that  performance  continues  to  be 
acceptable. 


Framing  Assumptions 

Our  research  in  this  area  explores  the  question  of  whether  identifying  framing  assump¬ 
tions  can  help  with  risk  identification  and  amelioration.  At  issue  is  whether  such 
assumptions  could  be  reasonably  identified  in  acquisition  programs.  We  defined  a 
framing  assumption  as  any  explicit  or  implicit  assumption  that  is  central  in  shaping 
cost,  schedule,  or  performance  expectations  and  identified  key  characteristics. 

•  They  drive  multiple  program  consequences  (with  respect  to  cost,  schedule 
performance). 

•  They  are  specific  to  the  nature  of  the  program  effort. 

Armed  with  this  definition  and  characteristics,  we  found  that  it  is  practical  to 
define  and  review  framing  assumptions,  even  early  in  a  program’s  life  cycle.  We  were 
able  to  establish  framing  assumptions  for  all  the  programs  despite  having  limited  access 
to  program  information.  Some  useful  observations  in  using  framing  assumptions  in  a 
risk  management  role  are  as  follows: 

•  Assumptions  should  be  kept  at  a  high  level  and  not  focus  on  minutiae  to  be  more 
useful  as  a  management  tool.  About  three  to  five  is  a  good  number  to  track  for 
most  programs.  However,  complicated  programs  or  those  with  a  rich  history  may 
become  challenging  to  reduce  to  a  few  key  assumptions. 

•  As  a  program  moves  through  the  acquisition  process,  framing  assumptions  may 
change  or  new  ones  emerge.  Therefore,  it  is  necessary  to  review  and  update  fram¬ 
ing  assumptions  periodically. 

•  For  programs  early  in  the  process,  it  is  relatively  easy  to  identify  a  narrow  set  of 
explicit  assumptions.  However,  it  is  difficult  to  uncover  implicit  ones.  For  mature 
programs,  it  is  more  complicated  to  identify  a  core  set  of  framing  assumptions 
because  of  the  wealth  of,  and  potentially  conflicting,  information. 
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•  It  is  easier  to  recognize  after-the-fact  failed  assumptions  than  it  is  to  recognize 
good  ones. 

How  successful  the  use  of  framing  assumptions  can  be  in  managing  risk  remains 
to  be  seen.  A  potentially  useful  extension  of  this  work  would  be  to  examine  fram¬ 
ing  assumptions  in  a  broader  range  of  programs  to  see  if  particular  assumptions  are 
problematic. 
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Concern  with  cost  overruns  in  major  defense  acquisition  programs  led  Congress 
to  direct  investigation  of  the  root  causes  of  overruns  in  programs  that  have 
breached  Nunn-McCurdy  thresholds.  The  authors  calculate  program  manager 
tenure  to  determine  whether  tenures  have  lengthened  since  policy  guidance 
was  issued  in  2005  and  2007.  They  also  address  the  question  of  whether  existing 
decentralized  systems  used  to  track  the  cost  growth  and  performance  of 
acquisition  category  II  programs  are  sufficient  or  whether  additional  centralized 
guidance  and  control  from  the  Office  of  the  Secretary  of  Defense  are  warranted. 
A  third  question  deals  with  the  management  of  cost  and  schedule  risk  and 
whether  the  identification  of  key  assumptions,  which  the  authors  call  framing 
assumptions,  could  be  a  useful  risk  management  tool. 
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